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INTRODUCTION 



THE PROBLEM 



How can the quality of education be improved? Quality 
education is defined here as teaching and learning which results 
in optimum student achievement and student attitude toward school 
subj ects. 



The major assumptions in this study were: 

1 Teaching and learning require effective interaction 
between teacher and students. 

2. Quality education greatly depends on teacher perform- 
ance in the classroom. 



3. Teacher effectiveness may be improved through 

organized feedback to teachers of observer ratings 
and pupil ratings of the teaching process. 

The above assumptions were the bases for the major 
hypothesis of this study. This hypothesis was that by increasing 
and systematizing feedback to teachers, the teaching process will 
be improved «rith the result that student learning will increase. 
Accordingly, the following paradigm was tested: 



TEACHER EFFECTIVENESS AND ATTITUDE TOWARD PUPILS 
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This study attempted to determine which of three types of 
feedback to teachers might result in a significant difference in 
student achievement and attitude toward courses of study. 
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OBJECTIVES 



The general objectives were to determine if student 
achievement and student attitude toward school subjects: 

1* ke improved significantly by systematically 

Incrstislng feedback to teachers concerning pupil 
reaction to their teaching. 

2. Cart be improved significantly by systematically 

increasing feedback to teachers concerning reaction 
of trained observers to their teaching. 

3 ^ Are significantly correlated with attitude of 
teachers toward their pupils. 

RELATED RESEARCH 



Kinhart (42) found that pupils of teachers who received 
ten hours of supervision did significantly better school work than 
pupils whose teachers were without supervision. Ten hours per 
month per teacher is costly. What is the minimum hours of teacher 
supervision which will make a significant difference in student 
growth? 



Costs of improving the quality of education could be 
reduced even further If significant improvement in student growth 
could be effected through pupil-teacher rating and feedback of 
results to teachers. Cage, Chatterjee and Runkel (40) found that 
sixth-grade teachers will modify their teaching in the light of 
Pupil rating if feedback of these ratings is given to the "teachers. 
If feedback of pupil reaction to the teaching of their teachers 
has as beneficial a result as feedback of observer ratings, then 
a considerable savings could be made in both time and money. 

Will high school teachers in various subject areas 
modify their teaching as a result of feedback of pupil ratings of 
their classroom performance? Will such modification of teaching 
significantly improve pupil achievement and attitudes? 



The valuable work of Gage and others suggests additional 
research. A better pupil-teacher rating scale may be needed since 
only four out of twelve items produced a statistically significant 
difference between teachers who received feedback and those who 
did not in the Gage Study. The Gage Study also suggested that a 
longer interval than 53 days may be needed between feedback of 
rating results and re— rating. The Gage Study did not include pupil 
achievement and pupil attitude toward subject with the experimental 
treatment . 
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There is evidence that students are honest, reliable 
raters of teachers and that students can furnish valuable evidence 
even though they are not experts on teaching. Bryan (5) cites 
studies of student ratings with reliability coefficients ranging 
from .64 to .99 with an increase in reliability as the studies 
move from the college level to the intermediate elementary level. 

Leeds (30) found that 100 teacher-pupil inventory scores 
correlated as follows: 

.43 with ratings by their principals. 

.49 with ratings by Leeds. 

.45 with ratings by their students. 

Evidence of validity of pupil ratings is that such 
ratings have correlated as follows with principals ratings for the 
same teachers : 

.60 with discipline ratings. 

.70 with teacher-pupil relations. 

.40 with ability to encourage learning (37). 

Starrak discovered that the opinions of students and of 
three faculty experts agreed 75 per cent of the time (35) • D ruckers 
and Remmers found that alumni rate their former teachers similarly 
to when, as students, they had rated the same teachers (24). 

At Brooklyn College no appreciable differences in 
instructors' scores were found to exist because of course grades 
received by students (6,681), size of classes, sex of students, 
college year, or whether or not the course was elective (26) . 

Remmers (34) and Hudelson (29) report low correlation coefficients 
(.07 and .19 respectively) between student ratings of college 
instructors and student grades received from their instructors. It 
appears that students, if approached properly, are honest, reliable 
raters of their teachers. 

Remmers also says : "If 25 or more student ratings are 

averaged, they are as reliable as the better educational and mental 
tests at present available.” (15) In this experiment the teachers 
were given their average ratings by item and for the total score. 

A great number of pupil-teacher rating instruments have 
been devised and are of value but it is indicated that a more 
satisfactory instrument is needed. Barr has examined many investi- 
gations of teaching efficiency of the past several decades and has 
found that insufficient attention has been paid to "...The partic- 
ulars of teaching..." (22) and the relationship between teacher 
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and student (23). Those devising teacher-rating instruments seem 
to have ignored the unidimensional scalogram technique. Yet 
McNemar conceded as early as 1946 that a scale devised by this 
technique has "...superiority on the single dimension problem..." 
(32) 



A unidimensional scale places each item along a single 
continuum similar to the inches of a yardstick. Each total score 
tells which items were reacted to favorably and which were reacted 
to unfavorably. Intensity analysis can be used to determine which 
score represents a dividing line between favorable and unfavorable 
attitudes. A satisfactory unidimensional scale is derived from the 
universe of attributes whxch define the concept and have a common 
content . 



Since there was evidence that the Hayes Pupil -Teacher 
Reaction Scale may be unidimensional, it was used in this study. 
Several administrations and successive refinements in 1960 of the 
Hayes Scale Indicated that there are certain desirable behaviors 
which are generally characteristic of poor teachers (41^. 

mi «i 

In 1961 a follow-up study (28) with the Hayes instrument 
resulted in what appears to be a unidimensional instrument to 
me jure attitude of students toward the teaching effectiveness of 
their teachers. The average coefficient f reproducibility was 
.93 for five instructors and 660 student raters. This average 
coefficient of reproducibility of .93 was an improvement of .22 
over the average minimum modal reproducibility. Intensity analysis 
for all 660 ratings indicated that an attitude content score of 
five was the fixed zero point dividing favorable from unfavorable 
attitudes. The attitudinal items were consistently rank ordered 
along a single continuum with the item marginals or proportions 
giving the favorable response ranging from .26 to .93. 

In the pr ;~eding study internal validity was established 
by adding eight statements ranging from "This instructor is the 
very best I've ever had" to "ThrH instructor is the worst I've ever 
had." The coefficients of correlation between scores on these 
eight statements and the scores for the attitude statements were: 

Instructor r Number of Raters 
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.93 


112 
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.85 


147 
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.70 


176 
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.81 


139 
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.65 


86 



In view of the advantages of a unidimensional scale that 
concentrates on classroom teaching it was considered desirable to 
use the Hayes Scale in this study. 
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Research on improvement of teaching indicates that 
teachers be observed while they teach. Medley and Mitzell note: 
Certainly there is no more obvious approach to research on teach- 
ing than direct observation c the behavior of teachers while thcy 
teach and pupils while they learn. Yet it is a rare study indeed 
that includes any formal observation at all," (14) 

Ryans offers this pertinent advice; 

Of the measurement approaches employing observation 
and assessment of teacher behavior in process only time 
sampling involving replic, ted systematic observation 
by trained observers produces sufficiently reliable 
data to recommend its use in fundamental research. . .the 
chief shortcoming of observation and assessment tech- 
niques has been lack of reliability, a shortcoming 
which recent research has indicated can fairly read- 
ily be overcome with care in definition and scale 
development, and with training of the observers or 
judges (16). 



Medley and Mitzell point out that validity of measurement 
of behavior depends on: 

1. A representative sample of the behaviors to be 
measured must be observed. 

2. An accurate record of the observed behaviors must 
be obtained. 

3. The records must be scored so as to faithfully 
reflect differences in behavior (14). 

Medley and Mitzell* s review of research and their own 
research led them to the conclusion that teacher observation should 
be separated from quantified rating. Since Anderson* s (21) pioneer 
work, others such as Bales, (2) Bush, (10) Thelen, (19) Withall, 
(38) Lewis, (31) Flanders, (39) and Amldon (20) have investigated 
student-teacher interaction as a determiner of effective teaching. 

The basic Flanders system involves ten categories for 
recording teacher-pupil-verbal-response behavior in the classroom^ 
Every three seconds the observer writes down the category number of 
the interaction he has just observed. These numbers ar*e recorded 
in sequence in a column and after the class are easily quantified 
into a matrix which provided insight into the teaching process. 
’’With the use of this recording procedure it is possible to assess 
the degree to which a teacher is meeting the social-emotional, as 
well as the intellectual, needs of the pupils." (4) 
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The Flanders instrument has been successfully field 
tested with 900 students, half in seventh-grade social studies and 
half in eighth-grade geometry. It was found that "...in both 
content areas the students of the more indirect teachers scored 
higher on achievement tests than did students of the more direct 
teachers." (1) It also found that the more flexible teachers who 
could shift readily from direct to indirect and vice versa approach- 
es had more success improving student achievement than did teach- 
ers w^th less flexibility in their classroom behavior. In view 
of the foregoing conclusion the Flanders system was selected for 
use in this study. 

The effect of feedback to teachers needs to be measured 
against the criterion of measured pupil gain, or end product. While 
a second rating after an appropriate interval subsequent to feed- 
back may reflect change in che teaching process, a primary criterion 
should be end product student measurement versus premeasurement. 

As Bloom (3) insists, "The research worker who wishes to under- 
stand teaching and tea' iers must understand not only the teaching 
and education process &s it takes place but also the outcomes or 
effects of the process — the change that takes place in the 
learners..." 



The Iowa Tests of Educational Development were selected 
to measure achievement. The average reliability of these tests is 
•91. Henry Chauncey had this to say about an earlier version of 
these tests: "...this program of tests i f probably the best 

conceived and executed battery that is available for use in the 
senior high school..." (7) Several years later Eric F. Gardner 
said: "As measures of certain broad aspects of the pupil 1 s 

educational development they are definitely superior tests." (8) 
More recently J. Murray Lee comments: "Lindquist, his associates 

and the publisher should be highly commended for the careful 
construction and standardization of the tests..." (9) 

Remmers' A Scale to Measure Attitude Toward Any School 
Subject was selected to measure pupil attitude in this area. The 
scale consists of seventeen items and columns for each subject so 
that students can indicate their attitudes by plus signs. Bonney 
anu Hampleman in commenting on this scale say: "It may be assumed 

that students' attitudes toward their school subjects are good 
indexes of group morale in these respective classes." (9) Newcomb 
commends Remmers for "...using clear and simple language iq his 
scaled statements..." (6) 

The investigator also felt the need to attempt to 
measure the attitude of teachers toward their pupils. Does rating 
feedback affect the way teachers feel toward pupils? Do classes 
which make the greatest gains have teachers whose attitude toward 
them as pupils is most favorable? 
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The Minnesota Teacher Attitude Inventory was selected to 
evaluate teacher-pupil relations. Persons scoring high on this 
inventory tend to be better teachers than persons scoring low. 
Dwight L. Arnold states: 

Two studies of validity were made; these studies 
yielded coefficients of correlation of .46 and .60 
between the scores on the test and the three criteria: 
principal's estimate, pupil's rating, and visiting 
experts rating. Consistent «nd thorough work has 
gone into construction and validation of this inventory. 
Clearly this represents a serious attempt on a very 
important and very difficult problem. ..It deserves.., 
extensive use in research. (8) 

Lee J. Cronbach also comments: 

Test development of exceptional quality lies behind 
this inventory. Publication was preceded by patient 
and careful research which is welll reported in the 
manual... The Minnesota Teacher Attitude Inventory is 
a promising tool for research on teacher success... (8) 

This study followed the recommendation of Ryans that 
studies be made "...of the influence of different in-service 
experiences of teachers..." (17) It attempted to answer criticism 
that educational research rarely seeks to improve the structure 
of what goes on in the classroom (13) . This study accepted the 
advice of those who say: "...it is with respect to the teaching 

process itself that the greatest potentiality for research lies.. 

." (33) This study was aimed at "doing something to change 
conditions rather than merely measuring and correlating them." (25) 
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METHOD 



GENERAL DESIGN 



Different kinds and anounts of feedback to teachers 
were used in an attempt to improve the teaching process and there- 
by improve pupil growth (paradigm on page one, this report). The 
independent variables were the various types of feedback to teachers. 
The dependent, criterion variables were: (1) end-of-year ratings 

and observations of teachers, (2) end-of-year teacher attitudes 
toward pupils, (3) end-of-year pupil achievement on standardized 
tests, and (4) end-of-year pupil attitude toward school subjects. 

The design may be diagrammed as follows: 



R 






R 









R 





In the above, R indicates random assignment of teachers 
and classes to one of four treatments and a control group. 

Oj, 0 ,., 0 ^, and 0 ^ indicates pre-measurement of achievement and 

attitude. X^, X£, X3, and X^, represent treatment conditions 
while O2, O4, Og, Og, and O^q represent post-tests for achieve- 
ment and attitude. 



POPULATION AND SAMPLE 



The typical Pennsylvania high school is similar to the 
typical American public high school. The typical American public 
high school has less than 100 seniors; 45% of Pennsylvania’s high 
schools have less than 100 seniors. The average American school 
system spends $532.00 per pupil (in average daily attendance) 
while the average school system in Pennsylvania spends $565.00 
per pupil. The average salary of the classroom teacher is $6,500. 
in the United States while the same average in Pennsylvania is 
$6,410.. The average high school class in the United States has 
approximately 23 pupils compared to 20 secondary pupils per class 
in Pennsylvania. In America the average school building is about 
24 years old while in Pennsylvania this average is about 30 years 
old. About 10 per cent of the tenth-grade students drop out of 
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school in both Pennsylvania and the nation. The average per 
capita personal income in 1964 for the United States was $2,500. 
while $2,575. was the case for Pennsylvania. In many respects 
the typical Pennsylvania high school represents the typical 
American public high school. 

In April of 1964 the 691 high schools of Pennsylvania 
with graduation classes were asked if they wished to participate 
in an experiment to improve teaching and learning. These schools 
were sent a four page summary of the proposed study. The 210 
schools which replied favorably were spread over a wide range by 
size and per pupil expenditure. To limit problems of coordination 
and socio-economic differences, it was decided to select from 53 
volunteering schools in south-central Pennsylvania serving rural 
neighborhoods containing no town of more than 8,000 people. From 
these 53 schools a random sample of 14 schools were selected with 
stratification of the sample by size and per pupil expenditure. 

The 14 selected high schools ranged in size from a school 
with a graduating class of 70 to one of 264. The percent of high 
school graduates going on to college ranged from 16 to 34 per cent 
in this sample of schools, while the average cost of instruction 
per pupil ranged from $315.00 to $506.00. 

To decrease problems of control and testing, this study 
was limited to one grade. The tenth-grade was selected because: 

(1) many students do not make much improvement in their scores on 
standardized English tests in high school subsequent to the tenth- 
grade and (2) comparatively few students drop out of school in 
Pennsylvania prior to the tenth-grade since school attendance is 
mandatory until age 17 unless a special work permit is approved. 

The four major subject areas of English, mathematics, 
science, and social studies were included to determine if feed- 
back would work equally well with teachers of different courses. 

No special education classes for children of limited ability were 
included. In the 14 randomly selected high schools, there were 
a total of 31 English teachers, 39 mathematics teachers, 27 
science teachers, and 25 social studies teachers. Twenty teach- 
ers from each of the four subject areas were randomly selected 
for the study and then randomly assigned as follows: 
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Treatment 1 


Number 


of Teachers by Subjects 
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English 1 


V 

Math 1 


V 

Science * 


Social * 
Studies ' 


Total 


1 


4 


4 
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16 


2 


4 
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16 
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4 
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16 
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16 
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4 


4 


4 


4 


16 


Totals 


20 


20 


20 


20 


80 



The random selection of the teachers was stratified by 
teacher salaries so that for each subject and for each treatment, 
one teacher was assigned from each of the following salary class- 
ifications: $4500 - $5199, $5200 - $5899, $5900 - $6499, and 

$6500 - 7400. This stratification by salaries also resulted in 
a reasonably good distribution of years of teaching experience 
among treatments since in this study teaching experience and 
salaries correlated .72. 

The total number of involved pupils was 2,186. This 
number sub-divided as follows: 471 in treatment 1, 439 in treat- 

ment 2, 464 in treatment 3, 41C in treatment 4, and 402 in treat- 
ment 5. 

The average I.Q. for the entire population was 110.94 
on the Science Research Associates Test of Educational Ability. 

The mean I.Q. 's for the 80 classes ranged from 91.79 to 129.25 
with nc significant differences among treatments or course subjects. 

Pupils could not be assigned randomly to treatments due 
to administrative difficulties. Entire classes were randomly 
selected from those taught by teachers who had been randomly 
selected and randomly assigned to treatments. Therefore class 
means were considered as the basic observations for determining 
degrees of freedom and analyzing achievement data. 

TESTING PROGRAM 



On August 30, 1965, the guidance counselors from the 
14 involved high schools attended an orientation meeting in the 
Pennsylvania Department of Public Instruction where adherence to 
standardized test directions and time limitations was stressed. 
During the third week of school these guidance counselors 
administered the following: (1) the Remmers Scale to Measure 

Attitude Toward Any School Subject, (2) Test of Educational 
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Ability (Science Research Associates), and (3) Form X-4, Iowa 
Tests of Educational Development. During the first week of May, 
Form Y-4, Iowa Tests of Educational Development were administered 
as a post-test. The Remmers Scale to Measure Attitude Toward 
Any School Subject was administered during April as another post- 
measurement. 

IN SERVICE EDUCATION 



During the second and third weeks of July 1965, the 
principal investigator and a co-investigator explained the study 
during visits to chief school administrators and high school 
principals of the 14 selected schools. On Saturday, September 
18, 1965, all of the involved 80 teachers attended an orientation 
meeting in the Department of Public Instruction. During this 
orientation meeting the following was done: (1) teachers 

completed the Minnesota Teacher Attitude Inventory, (2) the purpose 
and procedures of the study were explained by the principal invest- 
igator, (3) teachers worked a programmed learning approach to 
Flanders interaction analysis (Appendix A), and (4) Dr. Edmund J. 
Amid on and Dr. Anita Simon of Temple University explained how 
teachers could use the results of the Flanders system. 

TREATMENT 1 



In treatment 1, students rated their teachers four times 
in the fall (between October 1 and December 22) and twice in the 
spring (between March 11 and April 28) using the Hayes Pupil-Teacher 
Reaction Scale (Appendix B) . The fall ratings occurred three 
weeks apart. Feedback to teachers for the first two ratings 
occurred after the second rating, and on December 30, 1965, the 
teachers were given feedback for the last two fall ratings. Feed- 
back included class frequencies and averages for the various items 
and was furnished by mail directly to the teachers concerned 
(Appendix C) . On February 14, 1966, these teachers were mailed a 
su ...ary of teacher ratings to date (Appendix D). On March 11, 1966, 
or 10 weeks subsequent to the second feedback on December 30, 1965, 
the first spring cycle of ratings began. 

TREATMENT 2 



In treatment 2, teachers were observed four times in . 
the fall (between October 1 and December 22) and twice in the 
spring (between March 11 and April 28^ by two carefully trained 
observers using Flanders 10 categories for interaction analysis. 
The two observers were trained for five days by Dr. Edmund J. 
Amidon and Dr. Anita Simon at Temple .University. The observers 
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continued to praciicc with tapes of teachers' lessons until they 
achieved a consistent iiiter-observar reliability of .85 before 
beginning classroom visitations. Both observers always observed 
together and their results were averaged for feedback to the 
teachers. Within several days of each observation, feedback was 
mailed to each teacher (Appendix E) . The interval between the 
last fall feedback and the first spring observation was 10 weeks. 

TREATMENT 3 



Treatment 3 was the same as treatment 2 except that 
teachers were observed only twice in the fall (between October 1 
and November 11) and once in the spring (between April 1 and 

April 28). The interval between the last fall feedback and spring 
observation was 20 weeks. 

TREATMENT 4 



The teachers in this treatment on October 26, 1965, 
were mailed: (1) average class ratings on the Remraers Scale* to 

Measure Attitude Toward Any School Subject, (2) class rosters 

showing I.Q. and scores on the Iowa Tests of Educational Develop- 
ment (Appendix F). 

TREATMENT 5 



In treatment 5, there was no feedback of any type to 
teachers. However, in March, they were obser using inter- 
action analysis and also rated by their pupils. 

ADDITIONAL TREATMENT PROCEDURES 



In treatments 2, 3, and 4, teachers were rated by their 
pupils with the Hayes Scale twice in the fall (between October 1 
and November 11) and once in the spring (between April 1 and 
April 28) without feedback. In treatments 1 and 4, teachers were 
observed twice in the fall (between October 1 and November 11) 
and once in the spring (between April 1 and April 28) without 
feedback. In addition, one half of the teachers by subjects in 
treatments 1, 2, and 3 were randomly selected to receive pre- 
test data for the Remmers Scale, the Iowa Tests of Educational 
Development (ITED), and the Test of Educational Ability (IQ). 

This data was mailed to these teachers on October 26, 1965 
(Appendix F). On January 24, 1966, all teachers in treatments 
2 and 3 were mailed a letter with the subject "Interaction 
Analysis - Can It Help You?" (Appendix G). In May 1966 the 
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the principals rated the teachers on a graduated scale ranging 
from a low of 1 and high of 5. 

PROCEDURES FOR DATA ANALYSIS 



Since intact classes had been assigned randomly to 
treatments 9 the class attitude and achievement means were used 
as the basic observations and treatment effects were tested 
against variations in these means (11). Either analysis of 
variance or covariance was employed to test for significance. In 
reporting class means in Appendix H, the first two class means listed 
for treatments 1, 2 and 3 represent classes whose teachers received 
pre-test data; these classes comprised sub-treatments within the 
first three treatments. 

Cornell scalogram analysis (27) was performed for each 
cycle of ratings for the first nine items on the Hayes Pupil- 
Teacher Reaction Scale to determine the degree of unidimensionality 
of this instrument. Intensity analysis (18) was computed for 
the same items of this scale to determine the zero point dividing 
the population according to those with unfavorable and those with 
favorable attitudes toward the teaching of their teachers. 

Coefficients of correlations were also computed between 
the various variables in this study. 

TEACHER REACTIONS 



To obtain reactions of teachers to written feedback reports, 
three questionnaires (Appendix I) were designed and administered in 
April, 1966. FORM I was designed for teachers who received feedback 
on teacher-pupil verbal interactions. FORM II was prepared for 
teachers who received feedback on the Hayes Pupil-Teacher Reaction 
Scale. FORM III was designed for teachers who received IQ results and 
September results on the ITED and Remmers Scale. 

A follow-up interview with each teacher who received feed- 
back was made in May, 1966. Essentially the same questions were 
asked again as a reliability check of the responses to the question- 
naire and to permit clarification of teachers' reactions. Each 
interview was recorded on tape to assure accuracy in reporting. The 
responses obtained during the interviews are reported where there 
was disagreement between answers recorded on the questionnaires and 
those given by the teachers during the interview. 
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RESULTS 



INTELLIGENCE 



The class means for intelligence as measured by the Test 
of Educational Ability (published by Science Research Associates) 
are reported in Appendix H, Table 1. Classes *n the first three 
treatments were divided into six sub-treatments. Teachers in three 
of the sub- treatments received pre-test feedback while teachers in 
the remaining three sub-treatments did not receive pre-test feed- 
back. The first two means listed by subjects for treatments 1, 

2, and 3 represent classes whose teachers received pre-test feed- 
back. The foregoing procedure will be followed throughout Appendix 
H in recording class means. No significant differences in intelli- 
gence were found anong either treatments or sub-treatments (Appendix 
H, Tables 2 and 3). 

ACHIEVEMENT 



Student Achievement was measured at the beginning and at 
the end of this study by the Iowa Tests of Educational Development 
(ITED) . The ITED included the following: Test 1-Understanding 

Basic Social Concepts; Test 2-General Background in the Natural 
Sciences; Test 3-Correctness and Appropriateness of Expression; 

Test 4-Ability to do Quantitative Thinking; Test 5-Ability to 
Interpret Reading Materials in Social Studies; Test 6-Ability to 
Interpret Reading Materials in the Natural Sciences; Test 7-Ability 
to Interpret Library Materials; Test 8-General Vocabulary; Test 
9-Use of Sources of Information. For each of the 80 classrooms a 
composite score for English was obtained by combining results on 
Tests 3, 7, 8, and 9. Mathematics was represented by Test 4 only. 

A composite science score for each classroom was obtained for Tests 
2 and 6 and a composite social studies score was derived from Tests 
1 and 5. 



The pre- and post- composite class achievement means are 
shown in Appendix H, Tables 4, and 7. Analysis of variance (Appendix 
H, Tables 5, 6, 8, and 9) revealed no significant differences among 
these achievement means. 

In addition when each ITED Test was examined separately 
no significant achievement differences were obtained. 
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STUDENT ATTITUDE TOWARD SCHOOL SUBJECTS 



The pre-and post- test class attitude means were obtained 
from the Remmers Scale to Measure Attitude Toward Any School Subject 
(Appendix H, Tables 10 and 11) . Analysis of variance indicated 
significant pre-test differences among subjects and for interaction 
(Appendix H, Tables 12 and 14) but no significant post-test differ- 
ences (Appendix H, Tables 13 and 15) . 

The post- test class attitude means also were subjected 
to analysis of covariance, multiple classification, to equate the 
significant pre-experimental differences. The pre-test student 
attitude class means were used as the covariate. Analysis of 
covariance detected no significant differences at the .01 level 
in student attitude (Appendix H, Table 16) . 

STUDENT RATINGS OF THEIR TEACHERS 



The class means for the first and last cycle of student 
ratings are listed in Appendix H, Table 17. Statistical analysis 
of these means is reported in Appendix H, Tables 18, 19, 20, and 21. 
No significant differences were obtained. 

Cornell scalogram analysis was performed on the Hayes 
Pupil-Teacher Reaction Scale in this study. The response categories 
of the Hayes scale were dichotomized for scoring purposes prior to 
applying scalogram analysis. The justification for combining 
categories for scoring purposes was that verbal habits of people 
differ. The meaning of "extremely clear" to some is the equivalent 
of the meaning of "very clear" to others. These two categories 
could be combined for scoring purposes and given the weight of one. 
Likewise the meaning of "sometimes" might mean to some, what "seldom" 
means to others. These two categories could be combined and given 
a weight of zero. 

The rule followed to combine response categories was to 
draw a line between the responses so as to minimize the number of 
pupils in the low group above the line and the number of pupils in 
the high group below the line (12) . Item one was analyzed as 
follows. 
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Item Original Weights 


Tow 25% 


High 25% 


This teacher makes the 
lesson clear in the first 
few minutes of class. 








a Always 


3 


20 


310 


b Usually 


2 


129 


129 


c Sometimes 


1 


228 


8 


d Seldom or Never 


0 


72 


2 



Pupil responses were scored first using the above weights. 
Then, based on total scores, the lowest 25% of the pupils and the 
highest 25% were determined. Next, categories were combined so 
that responses "a" and "b" were recorded as "1" and responses "c" 
and "d" were recorded as "0" for this item. 

The same procedure was repeated for all items and the 
following scoring key resulted: 





Item 


Responses Scored as 1 


Responses Scored as 0 


1-4 


a, b 


Cj d 


5-6 


a 


b , c , d , 


7-8 


a, b 


c, d 


9 


a 


• b, c, d, — 



The above key resulted in a maximum possible score of 
nine or a minimum score of zero in content for any one student. 



Cornell scalogram analysis was then performed as follows 

(27): 

1. — The IBM cards were arranged into rank order by total scores 

from high to low. 

2. — A table was prepared with: 

a. One row for each person's score, using the rank order arrived 
at in step one above. 

b. One column for each category of each question. 

3. — The responses of each person were indicated on the table by 

placing a check (/) in the appropriate column for each item 
opposite the row representing the total score of that- person. 
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4. — Then the frequencies for each category were totaled. The sum 

of the frequencies in both categories for each question had to 
equal the total number of subjects answering the questionnaire. 

5. — Cutting points were placed for each item at the place in the 

rank order where the most common response changed from a favor- 
able category to a non-favorable one. Cutting points were 
established to minimize error. 

6. — The errors were totaled for each item using the cutting points 

to determine which responses fell outside the established patterns. 

7. — The coefficient of reproducibility for each item was computed 

by dividing the number of errors (for that item) by the total 
number of subjects and subtracting the result from one. 

8. — The marginals (per cent of people who answered each item favor- 

ably) were computed for each item. 

Appendix H, Tables 22 and 23 show a sample of the co- 
efficients of reproducibility and marginals resulting from scalogram 
analysis of the first two ratings in the fall. 

The marginals or percentage of the items answered favorably 
ranged from .24 to .76 with an average marginal of .50 which indicated 
that spurious reproducibility was not introduced by extremely high 
or extremely low marginals. For example, item one had a marginal 
of .76 which meant that the coefficient of reproducibility could 
not be less than .76 for this item. The coefficient of reproducibility 
for this item was .89 which represents a sound differer~e. 

Also the marginal for item eight was .24. This meant that 
the coefficient of reproducibility for this item could not fall below 
.76. It should be noted that a coefficient of reproducibility of 
.88 was obtained for this item. This represents a gain of .12 over 
the minimum obtainable reproducibility. The average minimum marginal 
reproducibility for all nine items was .63 which indicates that 
the average coefficient of reproducibility of .84 was not dependent 
on the marginals. 

When intensity analysis was performed a zero point was 
found which divided the sample into those with favorable and those 
with unfavorable attitudes. At the beginning of the year the zero 
point was 1, while at the end of the year the zero point was 3 
(Figures 1 and 2 on the following pages. 
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MEDIAN INTENSITY PERCENTILE 



FIGURE 1 

INTENSITY- CURVE FOR FIRST AND 
SECOND VISITATIONS 




MIDPOINT OF CONTENT PERCENTILE 



i 




FIGURE 2 

INTENSITY CURVE FOR LAST 
VISITATION 




MIDPOINT OF CONTENT PERCENTILE 
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The foldover technique was used to determine intensity 
scoring. The answers to each item were dichotomized. Students 
answering an item with "a" or "d" were scored as "1" and "0" for 
an item answered with "b" or "c". The rationale for this technique 
is that a student answering an item with "a" or "d" feels more 
intensely favorable or unfavorable toward an item than one who 
answers with "b" or "c". This technique meant that intensity 
scores ranged from zero to nine. 

The content scores (based on the number of favorable 
replies) were then recorded in a matrix. The cumulative percent- 
iles were computed for content scores and then the midpoints of 
content percentiles were computed. Next the cumulative percentiles 
for intensity scores were computed and then the median intensity 
for each content score was computed. The median intensity for 
each content score was then plotted against the corresponding 
midpoint for content percent ile (36) . 

Appendix H, Table 24 contains the intensity table for a 
combination of the. first and second visitation cycles, and Table 
25 contains the intensity scores for the sixth visitation cycle. 

INTERACTION ANALYSIS 
# 



The large I/D (ratio of indirect to direct teacher talk) 
and revised or small i/d (motivation versus control) spring ratios 
were investigated by analysis of covariance to determine the change 
in interaction as a result of the feedback process. The average 
fall means were used as covariates. Appendix H, Tables 27 and 28 
contain the ratios used in this investigation. 

No significant differences materialized for either the 
I/D or the revised i/d. Appendix H, Tables 29 and 30 report the 
results of the analysis. 

The I/D and revised i/d were tested for homogeneity of 
variance. The hypothesis of homogeneity was rejected at the .01 
level for both measures. A Kruskal-Wallis test was employed to 
analyze gain scores on the I/D and revised i/d measures. Again, 
no statistically significant differences were detected. 

Analysis of variance, two-way classification, by subiect 
and treatment failed to produce any statistically significant Je- 
suits when applied to measures taken from the matrices. These 
measures were: extended direct, extended student talk, revised 

i/d for rows 8 and 9, and the 3-3 cell. 
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MINNESOTA TEACHERS ATTITUDE INVENTORY (MTAI) 



The pre- and post-test scores of the MTAI (Appendix H, 
Tables 31 and 32) were examined by analysis of variance, two-way 
classification by treatments, sub-treatments and subjects. 

The analyses did not yield any significant results in 
either treatments or sub-treatments. Appendix H, Tables 33, 34, 
35, ai>d 36 comprise the findings of these analyses. 

CORRELATION COEFFICIENTS 



Coefficients of consistency for results on various 
administrations of the Hayes Pupil-Teacher Rating Scale range 
from .64 to .93 (Appendix H, Table 26). Additional correlations 
are contained in Appendix H, Table 37. 

Student achievement (ITED) and teacher attitude toward 
pupils (MTAI) correlated -.14 at the beginning of the school year 
and -.35 at the end of the school year. Student attitude toward 
srhool subjects (Remmers Scale) and student achievement (ITED) 
correlated .41 at the beginning of the school year and .18 at the 
end of the school year. 

Teacher attitude toward pupils (MTAI) and student 
attitude as measured by the Hayes Pupil- Teacher Scale correlated 
.28. Insignificant correlations were obtained between the student 
ratings of teachers and I/D ratios and also between student 
attitude toward school subjects (Remmers) and teacher attitude 
towaru pupils (MTAI) . 

TEACHER REACTIONS 



Teacher reactions to this study were investigated and 
are described in Appendix I by Floyd N. Keim, a co- investigator. 
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DISCUSSION 



The purpose of furnishing feedback by mail to teachers 
was to standardize this feedback and to present it in a clinically 
pure fashion uncontaminated by the human element as might be ex- 
pected in face-to-face or person-to-person feedback. A search of 
the literature and discussion with authorities in the field of 
interaction analysis indicated that this idea has not been tried 
previously. 

Since intact classes were randomly assigned to treatments, 
tue 80 class means for each criterion variable were used as the 
basic observations for statistical analysis and the degrees of 
freedom for total were only 79. The chances of obtaining statis- 
tically significant differences were accordingly much smaller than 
if the total number of pupils had been used to compute degrees of 
freedom and if each pupil's score had been used as a basic obser- 
vation in the statistical analysis. 

Attitude toward teachers was measured by the Hayes 
Pupil-Teacher Reaction Scale and no significant differences among 
treatment means resulted. The consistency in which students rated 
their teachers supported previous research that students are re- 
liable raters. The correlation between student rating of teachers 
and principals ' ratings of teachers indicated reasonable validity 
for the student ratings. Cornell scalogram analysis produced an 
average coefficient of reproducibility of .87 for the items on the 
Hayes Scale. Intensity analysis in the fall indicated that an 
attitude score of one was the dividing point between favorable and 
unfavorable attitudes toward the teaching of teachers. Intensity 
analysis in the spring indicated that an attitude score of three was 
the point dividing favorable from unfavorable attitudes as measured 
by the Hayes Scale. The average percent pro (favorable) was .50 on 
this instrument. The items on the Hayes Scale could be consistently 
rank ordered along a continuum with the item marginals or favorable 
response proportions ranging from .24 to .76. The Hayes Scale does 
appear to be unidimensional at the high school level. 

Intelligence, as measured by SRA Tests of Educational 
Ability was almost evenly distributed among treatments. The IQ 
treatment means were so similar that IQ was not used as a covariate 
in analyzing achievement results. Achievement as meat ired by the 
Iowa Tests of Educational Development also did not differ signif- 
ican tally among treatments; neither did the classroom interaction 
analysis patterns. 

Likewise, student attitude toward school subjects as 
measured by the Remitters' Scale did not indicate significant dif- 
ferences among treatment means. Teacher attitude means as measured 
by the Minnesota Teacher Attitude Inventory were lower in May than 
at the beginning of the school year but once again the differences 
among treatments were not significant. 

22 



CONCLUSIONS 



Iu this particular study where feedback to teachers was 
by mail only, student achievement and student attitude toward 
school subjects was not improved significantly by systematically 
increasing feedback to. teachers of either (1) pupil reaction to 
the teaching of their teachers or (2) results of classroom inter- 
action analysis using the Flanders* procedures. Also student 
achievement and student attitude toward school subjects were not 
found to be significantly correlated with attitudes of teachers 
toward their pupils (as measured in this study). 

The teacher questionnaire and interview results indicated 
that most of the teachers felt that they had gained many valuable 
ideas to improve their teaching but that personal, face-to-face feed- 
back would have helped them to a greater degree than did the written 
feedback via mail. • The Hayes Pupil-Teacher Reaction Scale does 
appear to be unidimensional at the high school level since the aver- 
age coefficient of reproducibility for All items was .87 and the 
items could be consistently rank ordered along a continuum with the 
item marginals ranging from .24 to .76. 



IMPLICATIONS 

It is indicated that this study did not provide sufficient 
in-service training for the teachers. In addition, face-to-face 
feedback by trained observers appears necessary to change teacher 
behavior and to improve teacher effectiveness. The problem which 
should be solved is how much and what kind of face-to-face feedback 
is needed to change teachers to the extent that teaching effective- 
ness is improved significantly. 

Also the Hayes Pupil-Teacher Reaction Scale should be used 
in other experiments to further check on its unidimensionality and 
to determine the conditions under which it can be used to improve 
teaching, learning, and the quality of education. 

Another recommendation to future investigators would be to 
arrange for feedback to teachers on the same day as observations and 
ratings occur. 



o 
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SUMMARY 



The general objectives of this study were to determine 
if student achievement and student attitude toward school subjects: 

1. Can be improved significantly by systematically 
increasing feedback to teachers concerning pupil 
reaction. 

2. Can be improved significantly by systematically 
increasing feedback to teachers concerning reaction 
of trained observers to their teaching. 

3. Are significantly correlated with attitude of 
teachers toward their pupils. 

In treatment 1 students rated their teachers four times 
in the fall semester. After these ratings each teacher was mailed 
a completed print-out of the average class ratings each time for 
each item and for total. Three months later the students again 
rated their teachers. If student rating of teachers could be used 
to improve teaching and learning, this would be a comparatively 
inexpensive way to effect the quality of education. 

Treatment 2 involved systematic recording of classroom 
teacher-pupil interaction (by two carefully trained observers) 
four times in the fall and twice in the spring. Treatment 3 was 
similar to treatment 2 except the teachers were observed only twice 
in the fall and once in the spring. In both treatments 2 and ? the 
interaction analysis results were mailed promptly and directly to 
the teachers. 

In treatment 4 the teachers were sent only the pre-test 
results for achievement and attitude toward school subjects. In 
treatment 5 the teachers received no feedback. In the first three 
treatments one-half of the teachers were also given pre-tests 
results but this did not apparently affect either achievement or 
attitude. 



Teachers in the first four treatments were asked to reply 
to a questionnaire and then they were interviewed to obtain addi- 
tional evidence concerning both the effectiveness of the study and 
how to improve future studies of a similar nature. Major suggest- 
ions of most teachers were more in-service training and face-to-face 
feedback rather than just written feedback by mail. 

No statistically significant differences were found be- 
tween achievement or attitudinal means. The Hayes Pupil-Teacher 
Reaction Scale analysis did indicate that it may be a unidimensional 
scale at the high school level. 
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APPENDIX A 



INTERACTION ANALYSIS: A PROGRAM FOR INTERPRETATION OF MATRICES 



Directions: See ’’Categories for Interaction Analysis” on last 

page and then return to Phase I below. 



PHASE I - INTRODUCTION TO MATRIX 



The observer becomes familiar with the classroom 
atmosphere before beginning to record interaction. Once the 
observer begins recording numbers, he categorizes the class- 
room interaction every three seconds or overy time a change 
in categories occurs. After the observer ceases recording, 
he has a list of category numbers which is the raw inter- 
action data. In order to interpret this data, he can 
produce a systematic summarization by entering the category 
numbers into a matrix. The matrix preserves the sequence 
pattern of events, but does not preserve the temporal order 
of events. 

1-1 The matrix could best be described as preserving: 

(a) Temporal order of events 

(b) Quantified sequence of events 

(c) Non-verbal behavior 



This is a Matrix 
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APPENDIX A (CONTINUED) 



The matrix consists of 10 rows and 10 columns. The 
rows run horizontally arid the columns run vertically. Each 
unit in the matrix is called a cell. 



l_l 


[ 1 1 


I 1 


! | 


1 1 


III] 


1 1 


1 1 


l ■ 1 



rt i i h 



row 



column 



1-2 A matrix consists of 10 



and 10 



- H- -f- 



cell 



1-3 Each unit in the matrix is called a 

1-4 The rows run and the columns run 



Filling in a matrix is called tabulating or tally- 
ing. After he has recorded his raw date (see A) in the 
classroom, the observer begins to tabulate the raw data into 
the matrix. To tabulate into a matrix, the observer must 
use the first number he has recorded as the row number, and 
the second number as the column number. For example, if 
the first number is "1" and the second number is "2", enter 
a tally mark into the matrix in row one, column two (see B). 
It is important to note that this pair of numbers is record- 
ed as one tally. 



Row 

Column 
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4 
8 
8 
2 
3 
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-H 



i 



ni 
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1-5 Filling in a matrix may be described as 



1-6 In a consecutive pair of numbers, the first is called 
the number . 



Rows and col- 
umns 
Cell 

Horizontally 
and vertically 



Tabulating or . 
tallying 

Row 



1-7 The second number is called the 



number „ 



Column 
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1-8 Each tally in the matrix represents recorded 

numbers. 

1-9 If a consecutive pair of numbers are 2,4 then the row 
number is and the column number is 



1-10 The cell in which the above pair of numbers is placed 
is called the cell. 



In the example of a segment of classroom inter 
action (see C) we notice that each number is used twice. 





C 

1 


Row 


Row 


2> 


Column 


Column 


% 


Row 


Row 


Column 


Column 


<8 


Row 


Row 


( 8> 

c 2 


Column 


Column 





1-11 The column number of the first pair becomes the 

number of the second pair. 

1-12 Using the same procedure, we can see that the third 

pair of numbers has as its row number the number 

and as its column number the number 



1-13 The fourth pair of numbers has as its row number the 
number and as its column number the number 



1-14 The tabulation or tally for the fourth pair of numbers 
would therefore be places in th e c ell. 

PHASE II - APPLICATION OF RAW DATA TO MATRIX 

(refer to figure 1) 



We shall now construct, a matrix from a summar- 
ization of data obtained from a 15 minute class period. 
Example: Teacher: "John, What day of the week is it?" 

John: "Today is Friday." 

This dialogue would be categorized as a 4-8 and one tally 
or tabulation would be placed in the 4-8 cell. 

2-1 The 4-3 cell is located at the intersection of 

4 and 8 . 



2 

2,4 

2,4 



Row 

3,4 

4,8 

4,8 



A-3 



Row 

Column 



APPENDIX A (CONTINUED) 



2-2 If there were 16 such Interactions during the class 

period, we would place the number in the 

4-8 cell. 



Directions: Insert the given number of tallies in their 

proper cells to complete the practice matrix, 
(figure 1) 



2-3 The teacher has interrupted his lectures with questions 

9 times. Tne 9 would be place in row column 

. Place 9 tallies in the 5,4 cell. 

2-4 The teacher has continued to lecture after asking a 

question 4 times. These tallies belong in row 

column . Place 4 tallies in the proper cell. 

2-5 The teacher has asked extended or long questions 11 

times. These tallies go in row _column 

Enter the tallied in the proper cell. 

2-6 The students have responded to direct questions with a 
narrow, factual answer 44 times. You can summarize this 

particular interaction by placing the number 

in row 4 column 8. Enter the tallies in the proper cell. 

2-7 There were no responses to teacher questions in which 
the students presented their own ideas. Therefore there 
are_ tallies in cell 4-9. 

2-8 A factual student response wes followed 29 times by 
teacher priase or encouragement. The number 29 goes 

into the cell. Enter the tallies in the proper 

cell. 

2=9 A factual student response is followed by a teacher 
question 13 ‘ imes. The number 13 wculd be place in 

row column . Enter the tallies in the 

proper lell. 

2-10 A factual student response is followed by teacher direc- 
tions 12 times. The number 12 is placed in the 

cell. Enter the tallies in the proper cell. 
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INCOMPLETE PRACUCE MATRIX 
Figure 1 
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PHASE III - INTERPRETING CLASSROOM INTERACTION FROM THE "FRINGE" 

MATRIX DATA 



We now have an idea of how raw data is entered 
into the matrix. We can now quantitatively analyze the 
data that we have in the matrix by working with the "fringe" 
or border areas of the matrix, (refer to figure 2) 

The first data that can be taken from the fringe 
area is the percentage of the total interaction in each 
column. This is found by dividing the number of tallies in 
each column by the total number of tallies which in this 
case is 343. (see figure 2) 

3-1 The greatest proportion of interaction is found in 
column • 



3-2 The percentage of interaction in column 8 is 

3-3 The smallest proportion of interaction is found in 
column and . 
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3-4 The percentage of interaction in column 1 is 
In column n the percentage is also 



Referring to figure 2, we notice that we have 
calculated percentages for each column. From the column 
percentages we can find the percentage of "teacher talk", 
by adding columns 1-7= 



3-5 The percentage of teacher talk is . This means 

that the teacher talked 66% of the time in that class. 



66 % 



3-6 We can find percentage of student talk by adding 
columns , 



8,9 



3-7 The percentage of student talk is 



28% 



3-8 The percentage of total talk in the classroom can be 
found by addin g 



rows 1-9 or 
teacher talk 
plus student talk 



In order to determine the nature of the teacher's 
influence on the students, we calculate the ratio of the 
indirect teacher influence (sharing, praising, accepting, 
questioning) to the direct teacher influence (lecturing, 
commanding, criticizing). This is called an I/D Ratio. 

The indirect teacher influence columns are 1 through 4 
and the direct teacher influence columns are 5 through 7. 



I/D * 1+2+3+4 
5+6+7 

To calculate the total number of indirect tallies 
we add columns 1 through 4. 



3-9 The total number of indirect tallies in the sample 

matrix is . 113 

3-10 The total number of direct teacher influence tallies 

in the sample matrix is . H2 



3-11 If we divide the indirect tallies by the direct tallies 

we get an_ ratio of . For example, if a 

teacher has 213 tallies in rows 1-4 and 5-7, this would 
give an I/D ratio of 213 which equals 1. 

213 



I/D, 1.008 
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An I/D ratio of 1.00 ineans that for every indirect 
teacher influence statement, there was one direct teacher 
influence statement. 

3-12 A teacher had 200 indirect statements and 100 direct 
statements. His I/D ratio is 

3-13 An I/D ratio of 2.00 would mean that for every two 

indirect teacher influence statements, there was 

direct teacher influence statement. 

3-14 An I/D ratio less than 1.00 would mean that the teacher 

is more than indirect in his influence on the 

students. 

3-15 Therefore a more indirect teacher would have a 

I/D ratio than* a more direct teacher, (a) higher, 

(b) lower 



A revised I/D ratio, sometimes written as small 
i/d ratio, can be used to determine teacher emphasis on 
motivation and control. In this ratio, the number of 
tallies in columns 1, 2, 3, is divided by the number of 
tallies in columns 6 and 7. 

i/d = 1+2+3 
6+7 

3-16 The revised i/d differs from the I/D ratio in that we 
omit columns and 



3-17 The revised i/d ratio is less concerned with actual 

presentation of subject matter and more concerned with 

and 



3-18 The revised i/d ratio for our practice matrix is . 

3-19 The above revised i/d ratio indicates that the teacher 

used more _teacher influence tha n teacher 

influence in motivating and controlling the students 
because the i/d ratio is less than 1. 

3-20 If the same teacher would have had a revised i/d larger 

than one, then we would say that in matters of motivation 
and control, the teacher’s influence was usually . 



« 
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indirect 

indirect 
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Our final I/D is called the I/L ratio for rows 8 
and 9 or I/D 8,9. This .ratio is found by adding the tallies 
in rows 8 and 9, columns 1 through 4 and dividing this 
number by the tallies in rows 8 and 9, columns 5 through 
7. The type of teacher reaction, direct or indirect, to 
student talk is recorded by thi 3 ratio. 

3-21 In columns 1-4 the tallies in rows 8-9 represent . 

(a) The direct portion oi rows 8 and 9, (b) The in- 
direct portion of rows 8 and 9. 

3-22 In columns 5-7 the tallies in rows 8-9 represen t . 
(a) The direct portion of rows 8 and 9, (b) The in- 
direct portion of rows 8 and 9„ 

3-23 I/D 8,9 for the practice matrix is . 

3-24 The I/D 8,9 gives us some idea of the type of teacher 
response to student . 

3-25 The high I/D 8,9 for our practice matrix would indicate 

that the teacher uses influence more than 

i nfluence in his responses to student talk. 



(b) 

(a) 

47 - 2.31 
20 

Talk 

Indirect , 
direct 



STU- DIRECT . INDIRECT 
DENT INFLUENCE INFLUENCE 
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COMPLETE PRACTICE MATRIX ’’FRINGE" DATA 

Figure 2 
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Total Talk = 94% 
Teacher Talk =66% 
Student Talk =28% 



I/D = 1.008 
i/d = .68 
I/D 8,9 = 2c 31 



A-9 



APPENDIX A (CONTINUED) 

PHASE IV - INTERPRETING INTERACTION FROM THE "CORE" OF THE 
MATRIX 



Now that we have examined the interpretations that 
can be made from the "fringe" areas of the matrix, we can 
turn to interpretations that can be made from the buildup 
of tabulations in individual ceils and certain areas within 
the matrix. For this purpose we will use figure 3 which 
outlines particular areas of the matrix with which we are 
concerned. We can then refer to the data in figure 2 to 
give these areas practical significance. 



If you will look at figure 3, you will see a shaded 
area in the form of a cross in rows 4 and 5 columns 4 and 5. 
This area is called the content cross. Tallies in this area 
represent teacher lecture or teacher asking for information. 
Therefore, this area indicates teacher emphasis on subject 
matter. 

4-1 The "content cross" covers rows a nd horizontally 

and columns and vertically. 

4-2 We also notice that there is an area A at the inter- 
section of the cross. This area shows us the amount of 
extended teacher and 



4-3 Turn to figure 2. Looking at area A, we notice that 

the 5-5 cell contains 18 tallies. This means that there 
were 18 pairs of 5-5 combinations or 18 instances of 
extended 



4-4 An interaction such as this, lecture followed by lecture 
or praise followed by praise, is called as "steady-state.' 
Therefore the 5-5 cell would be called a__ cell. 

4-5 Another "steady-state" cell in the "content cross" is 
the cell. 



4-6 Are there any other "steady-state" cells within the 
"content cross?" (a) Yes, (b) No 
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There are ten "steady-state" cells in the entire 
matrix. These are the 1,1;2,2; etc. through 10, 10 cells 
from the upper left diagonally to the lower right of the 
matrix. Entries in "steady-state" cells indicate that the 
speaker has remained in a particular category for more than 
3 seconds. All other cells are "transition" cells. 

4-7 The 5-4 cell is cell. 



4-8 The only other "transition" cell in area A is r.he 
cell. 



Transition 



4-5 



4-9 The 5-4 cell shows a transition from teacher lecture 
to teacher 



Question 



4-10 The 4-10 cell shows a transition from to 



Teacher 
question to 
silence 



Focusing our attention to the right-hand side of 
the matrix in figure 3, we see area B. This is the area of 
student response. By looking at the tallies in the separate 
cells in columns 8 and 9, rows 1 through 7, we can tell what 
stimulated or initiated student talk. 



4-11 In order to determine what initiated student talk, we 
should look at columns and , rows through 



8,9: ;1,7 



4-12 The 44 tallies in the 4-8 cell in our practice matrix 
(figure 2) indicate that 44 times student talk was 
initiated by teacher . 



Questions 



4-13 The 44 tallies in the 4-8 cell indicate memory-type 
teacher followed by factual, narrow student 



Questions, 

Answers 



4-14 The 23 tallies in the 6-8 cell indicate teacher 
followed by student . 



Directions, 

responses 



4-15 The 8-8 cell is a .cell. 

4-16 The 28 tallies in the 8-8 cell indicate amount of 

student responses that were longer than seconds in 

duration. 



"steady-state" 
or extended 
student talk 
3 
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Just as area B in figure 3, shows what initiated 
student talk, area C (rows 8 and 9, columns 1 through 7) 
shows the type of teacher response that followed student talk. 

4-17 Since all of the student responses in our practice 

matrix (figure 2) were factual, narrow responses, 11 
of the tallies in area C are in row . 

4-18 In figure 2 the teacher has usually responded to 

student answers with praise. We find this by looking 
at cell 



4-19 Cell 8,2 has 



4-21 



tallies. 



4-20 Cell 8,3 has 



tallies. 



8 



8-2 

29 

5 



If we compare .cells 8-2 and 8-3 in fig e 2, we would 

say that the norm for this teacher is . (a) The 

teacher uses student ideas, elaborating and expanding 
on them and integrating them into the lesson. (b) The 
teacher praises the students for giving the predetermined 
answer and then continues x^ith his lesson. 



Moving next to the area designated as D in figure 3, 
which is referred to as the extended direct: area, we have cells 
showing criticism — directions sequences. This area also shows 
v he vicious circle sequence in x^hich the teacher gives a 
direction, the students resist, the teacher criticizes and gives 
another direction, etc. 



4-22 Tallies in the "vicious circle" reflect upon_ 



(a) 



classroom management and con rol, (b) subject matter 
content 



4-23 The practice matrix (figure 2) shows that 



. (a) 

students did resist directions and x^ere criticized* 
(b) students did not resist directions 
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MATRIX CODE AREAS 
Figure 3 




CONTENT 

CROSS 



4-24 The term "vicious circle" . (a) would apply to 

the practice matrix (figure 2) , (b) would not apply to 
the practice matrix (figure 2) 

4-25 Area E in figure 3, shows us the amount of extended 
teacher influence, (a) indirect, (b) direct 



4-2b The tallies in the 2-3 cell indicate a transition in 

teacher behavior fro m t o of student ideas. 

4-27 Judging from the number of tallies in the 3-3 cell in 

figure 2, we can assume that teacher acceptance and use 

of student ideas occurred . (a) frequently, (b) 

infrequently, (c) never 



b 
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Praise, 

acceptance 
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Figure 4 



TEACHER II 



8 



8 



23 



22 



9 



27 



35 



14 



INDIRECT 

INFLUENCE 



5Q. 

21 



2_ 

1 



DIRECT 

INFLUENCE 



22 



26 



52 



22 13 



8 



10 Total 



31 



STUDENT 

TALK 



Total Talk = 97% 
Teacher Talk = 62% 
Student Talk - 35% 



I/D = 1.20 
i/d = 2.8 
I/D 8,9 - 2.06 
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PHASE V - PROBLEMS INVOLVING THE COMPARISON OF TWO MATRICES* 
FINAL EXAM 



5-1 Which cell is used to determine extended teacher lecture 7 
(a) 4-4, (b) 5 m5, (c) 6-6 



In working the following problems, you will compare our 
practice matrix (figure 2, teacher 1) with the sample matrix 
(figure 4, teacher 2) . 



5-2 Which teacher, comparing teacher //I with teacher #2, has 
more extended lecture? 

(a) teacher 1, (b) teacher 2 

5-3 Which teacher is the more indirect? 

(a) teacher 1, (b) teacher 2 

5-4 Does an indirect teacher necessarily spend less time on 
extended lecture? 

(a) yes, (b) no 

5-5 Which teacher asked more narrow, factual questions which 
were followed by factual student replies? 

(a) teacher 1, (b) teacher 2 

5-6 Which teacher used more extended commands or directions? 
(a) teacher 1, (b) teacher 2 

5-7 Which teacher has more lecture followed by student talk? 
(a) teacher 1, (b) teacher 2 

5-8 Which teacher is more inclined to encourage or praise a 
student response? 

(a) teacher 1, (b) teacher 2 

5—9 In which ratio do these two teachers differ more? 

(a) I/D, (b) revised i/d 

5-10 This indicates a greater difference between the two in 
method of: 

(a) content presentation, (b) motivation and control 
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^-11 Immediately following student talk which of the two 
teachers responds more indirectly? 

(a) teacher 1, (b) teacher 2 

5-12 The answer to question 5-11 is determined by the: 

(a) I/D ratio, (b) I/D g ^ ratio, (c) revised i/d ratio 

5-13 From the I/Dg 9 ratio we can say that: 

(a) both teachers respond more indirectly than directly 
to student talk. 

(b) only teacher 1 responds more indirectly than 
directly to student talk. 

(c) only teacher 2 responds more indirectly than directly 
to student talk. 

(d) both teachers respond directly to student talk. 

5-14 The big difference between the two teachers in area C 
(teacher response to student comments) can be seen by 
comparing: 

(a) the total numoer of tallies in the area for both 
teachers 

(b) row 9 of both teachers, (c) the 8-8 cell of both 
teachers 

5-15 Looking at area 13, we can see some evidence of the 
"vicious circle" in: 

(a) teacher 1, (b) teacher 2, (c) neither 

5-16 Ir area E (extended indirect influence) we can see that 
teacher 2 has many more tallies than teacher 1, out *ne 
significant difference in this area can be seen in the. 

(a) 2-2 cell, (b) 2-1 cell, (c) 3-3 cell 

5-1.7 Judging from the two matrices and from what you have 

learned , which teacher appears to be the more flexible 
in his teaching behavior? 

(a) teacher 1, (b) teacher 2 

5-18 In comparing the two matrices, it would seem that the 

students in which class had a greater freedom of speech 
and were not afraid to present their ideas to the class? 

(a) teacher l's class, (b) teacher 2 f s class 

5-19 Which teacher asked a question that was followed by 
silence? 

(a) teacher 1, (b) teacher 2 

5-20 Which teacher uses questions more often during his lectures? 
(a) teacher 1, (b) teacher 2, (c) both use questions equally 

during lecture 
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ANSWER SHEET 



b 


5-11 


b 


5-12 


b 


5-13 
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5-14 


a 


5-15 
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5-16 
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5-i8 
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b 
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SUMMARY OF 

CATEGORIES FOR INTERACTION ANALYSIS 



-? 




1 . 




INDIRECT 


2 . 




INFLUENCE 


3 . 


TEACHER 1 






TALK 




4 . 






5 . 




DIRECT 


6 . 






7 . 




INFLUENCE 








8. 


STUDENT 


9 . 


TALK 






10 . 



* ACCEPTS FEELING: accepts and clarifies 

the feeling tone of the students in a 
nonthreatening manner. Feelings may be 
positive or negative. Predicting and 
recalling feelings are included. 

*PRAISES OR ENCOURAGES: praises or en- 

courages student action or behavior. 

Jokes that release tension, not at the 
expense of another individual, nodding 
head or saying "uhhuh?" or "go on" are 
included. 

*ACCEPTS OR USES IDEAS OF STUDENT: 
clarifying, building or developing ideas 
or suggestions by a student. As teacher 
brings more of his own ideas into play, 
shift to category five. 

*ASKS QUESTIONS: asking a question about 

content or procedure with the intent that 
a student answer. 



^LECTURES : giving facts or opinions about 

content or procedure; expressing his own 
idea; asking rhetorical questions. 

*GIVES DIRECTIONS: directions, commands or 

orders with which a student is expected 
to comply. 

^CRITICIZES OR JUSTIFIES AUTHORITY : 
statements, intended to change student 
behavior from nonaceeptable to acceptable 
pattern; bawling someone out; stating why 
the teacher is doing what he is doing, 
extreme self-reference. 



* STUDENT TALK-RESPONSE: talk by students 

in response to teacher. Teacher initiates 
the contact or solicits student statement. 

^STUDENT TALK-INITIATION: talk by students, 

which they initiate. If "calling on" 
student is only to indicate who may talk 
next, observer must decide whether student 
wanted to talk. If he did, use this category. 

TSnSdTSToSSiSsim : pauses, short periods 

of silence and periods of confusion in which 
communication cannot be understood by the 
observer. - 
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HAYES PUPIL-TEACHER REACTION SCALE 



DIRECTIONS 



Please read these Instructions first. Your sincere, thoughtful help 
will be appreciated. 



Do not place any marks on this paper. On the IBM card print your name 
(last name first), your teacher's name, school name and today's date. 
Then indicate your reaction to each item on the IBM card with the 
electrographic pencil which has been provided. In using the electro- 
graphic pencil, please darken heavily the entire space for each 
answer which you select. 



YOUR TEACHER AND SCHOOL ADMINISTRATORS WILL NOT SEE YOUR ANSWERS . The 
reaction of your entire class (a r a group) will be given directly to 
your teacher by the Harrisburg Research Team. 



DO NOT TURN THE PAGE UNTIL TOLD TO DO SO . 
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HAYES PUPiL-TEACHER REACTION SCALE 



1. This teacher makes the lesson objectives clear in the first 
few minutes of the class. 4 



a. 

b. 

c. 

d. 



2. She (he) really 

a. 

b. 

c. 

d. 



Always . 

Usually. 

Sometimes. 

Seldom or never. 



causes you to think: 

v 

Most of the time. 
Often. 

Sometimes. 

Seldom or never. 



3. Her (his) explanations are: 

a. Extremely clear and to the point. 

b. Very clear and to the point. 

c. Adequate, might be better. 

d. Often not clear or not to the point. 



4. Her (his) lesson materials are: 

a. Outstanding. 

b . Very good . 

c . About average . 

d. Definitely below average. 

5. Her (his) lessons provide very well for the needs, interests, 
and experience level of students: 

a . Always . 

b. Usually. 

c. About half the time. 

d. Sometimes or seldom. 
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HAYES PUPIL-TEACHER REACTION SCALE. 



6. Her (his) instruction is very realistic: 

a. Always. 

b. Often. 

c. Sometimes. 

d. Seldom or never. 



7. Her (his) instruction is: 

a. Extremely challenging. 

b. Very challenging. 

c. Somewhat Challenging. 

d. Nut very challenging or usually unchallenging. 



8. She (he) concludes lessons by: 

a. Capably emphasizing the main points. 

b. Repeating the main points. 

c. Abruptly stopping, but this does not bother me. 

d. Abruptly stopping and I often wonder what I 
should have learned during the period. 



9. This teacher uses excellent examples to make ideas clear: 

a. Most of the time. 

b. Usually. 

c. About half the time. 

d. Sometimes or seldom. 



10. This instructor is the very best I've ever had. 

a. Agree 

b. Disagree 



11. This instructor is one of the best. 



a. Agree 

b. Disagree 



12. Her (his) teaching is effective. 

a. Agree 

b . Disagree 
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HAYES PUPIL-TEACHER REACTION SCALE 



13. Her (his) lessons are at least average or better. 

a. Agree 

b. Disagree 

14. Her (his) teaching is not quite adequate. 

a. Agree 

b. Disagree 

15. Her (his) lessons are poor. 

a . Agree 

b. Disagree 



16. This teacher is very ineffective. 

a. Agree 

b. Disagree 



17. This teacher is the worst I*ve ever had. 

a. Agree 

b. Disagree 



18. This subject s one of the best. 

a. Agree 

b. Disagree 



19. This subject is "okay". 

a. Agree 

b. Disagree 



20. This subject is dullo 



a . Agree 

b. Disagree 
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Teacher 



School 



Date of Pupil Completion of Hayes Reaction Scale 



Rationale for Pupil Teacher Rating 

Kinhart^ found that pupils of teachers who received ten hours of 
supervision did significantly better school work than pupils whose 
teachers were without supervision. Ten hours per month per teacher is 
costly. A need exists to determine if significant improvement in 
student growth can be achieved with the minimum hours of teacher 
supervision. 



Costs of improving the quality of education could be reduced 
even further if significant improvement in student growth could be 
effected through pupil teacher rating and feedback of results to 
teachers. Gage, Chatterjee and Runkel^ found that sixth grade teachers 
will modify their teaching in the light of pupil rating if feedback of 
these ratings is given to the teachers. If feedback of pupil reaction 
to the teaching of their teachers has as beneficial a result as feedback 
of observer ratings, then a considerable savings could be made in both 
time and money. 



Teachers need to see themselves as their students see them. The 
crucial test of teaching is how it changes the students who are being 
taught. Students are the only group who see their teachers day in and 
day out in the classroom. Students are not experts on the teaching 
process- but they can furnish valuable evidence which should be used to 
improve teaching. Objective teachers want to know how they ean get 
along more successfully with their students . 



There is evidence that students are honest, reliable raters of 
teachers and that students can furnish valuable evidence even though 
they are not experts on teaching. Bryan^ cites studies of student 
ratings with reliability coefficients ranging from .64 to .99 with an 
increase in reliability as the studies move from the college level to 
the intermediate elementary level. 
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Evidence of validity of pupil ratings is that such ratings 
have correlated as follows with principals * ratings for the same 
teachers: 

.60 with discipline ratings 

.70 with teacher-pupil relations 

.40 with ability to encourage learning. ^ 

Starrak discovered that the opinions of students and of 
three faculty experts agreed seventy-five percent of the time. 5 
Druckers and Rcmmers found that alumni gave similar ratings to 
their former teachers when these alumni had been students. 6 

At Brooklyn College no appreciable relation was found to 
exist between ratings given to teachers by students and each of the 
following: course grades received by students (6,681), size of 

classes, sex of students, college year, or whether or not the course 
was elective, 7 Remmers^ and Hudelson^ report low correlation coeffi- 
cients (.07 and .19 respectively) between student ratings of college 
instructors and student grades received from their instructors. It 
appears that students, if approached properly, are honest, reliable 
raters of their teachers. 

Rammers also says: 'If 25 or more student ratings are averaged, 

they are as reliable as the better educational and mental tests at 
present available. ”10 In this experiment you are given your average 
ratings by item and for total since a few students can be expected to 
be much too far to one extreme or another. 



Gage, Runkel, and Chatter jee found that when sixth grade teachers 
were furnished information on how their pupils described their actual 
teacher and how they described their ideal teacher, teachers* behaviors 
changed in the direction of their pupils* conception of an ideal teacher 
by the second time the pupils rated their actual teacher a month or two 
later. The feedback not only changed teacher behavior but the teachers 
were able to more accurately predict their pupils* description of their 
teaching. 

In this study students will rate their teachers in the fall 
semester with feedback to certain randomly sleeted teachers. In March 
and April students will rate their teachers so the investigator may see 
if teachers have improved in the eyes of their students. 

A great number of pupil-teacher rating instruments have been 
devised ai»d are of value but it is indicated that a more satisfactory 
instrument is needed. Barr has examined many investigations of teaching 
efficiency of the past several decades and he has found them paying 
insufficient attention to r, ...The particulars of teaching. . .*‘11 and 
the relationship between teacher and student, 12 Those devising teacher- 
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rating instruments seem to have ignored the unidimensional scalo- 
gram technique. Yet McNemar conceded as early as 1946 that a 
scale devised by this technique has "...superiority on the single 
dimension problem. . ."13 

A unidimensional scale places each item along a single 
continuum similar to the inches of a yardstick. Each total score 
tells which items were reacted to favorably and which were reacted 
to unfavorably. Intensity analysis can be used to determine which 
score represents a dividing line between favorable and unfavorable 
attitudes. A satisfactory unidimensional scale is derived from the 
universe of attributes which define the concept and have a common 
content . 

Since there is evidence that the Hayes Pupil-Teacher Reaction 
Scale may be unid linens ional, it will be used in this study. Several 
administrations and successive refinements in 1960 of the Hayes Scale 
indicated that .here are certain desirable behaviors which are 
generally characteristic of good teachers and that these behaviors 
are not generally characteristic of poor teachers . I* 4 

In 1961 a follow-up studyi5 with the Ilayes instrument resulted 
in what appears to be a unidimensional instrument to measure attitude 
of students toward the teaching effectiveness of their teachers. In- 
tensity analysis for all 660 ratings indicated that an attitude content 
score of five was the fixed zero point dividing favorable from unfavor- 
able attitudes. The attitudinal items were consistently rank ordered 
along a single continuum. 

In the preceding study internal validity was established by 
adding eight statements ranging from "This instructor is the very best 
I've ever had" to "This instructor is the worst I've ever had." The 
coefficients of correlation between scores on these eight statements 
and the scores for the attitude statements were: 



Ins true toi 


r 


Number of Raters 


A 


.93 


li *» 


B 


.85 


14. 


C 


.70 


176 


D 


0 81 


139 


E 


65 


86 



In view of the advantages of a unidimensional scale that concen- 
trates on classroom teaching, it would be desirable to see if it will 
work in this study. A copy of this scale is enclosed. Also the 
reactions of your pupils to your teaching is indicated on the attached 
sheets . We hope that by your analysis of these results you will get 
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some ideas which will be of help to you. 

Frequency 

Items of Answers 

1, (Lesson objectives clear) 

a. __ 

b • 

c, ' 

d. 



2, (Causes students to think) 

a. 

b. 

c. 

d. 



3. (Clarity of explanation)* 

% 

a. 

b. 
c « 
d. 



4. (Quality of lesson materials) 

a. 

b. 

c. 

d. 



5. (Provision for student: needs) 

a. 

b . 

c. 

d. 



6. (Instruction is realistic) 

a. 

b. 

c. 

d. 



7. (Challenging teaching) 

a. 

b. 

c. 

d. 



Class 

Average 



C-4 



APPENDIX C (CONTINUED) 



Teacher 



8. (How lesson Is concluded) 

a. 

b. 

c. 

d. 

9. (Variety of methods) 

a. 

b. 

d. 



On the above items your score was o ut of a possible 9. 

• For statements. 10 through 17 your average falls in ite m . 

percent of your class agreed that the subject was 

one of the best. 

percent of your class said the subject "okay”. 

Please be assured that no one except yourself has been furnished 

this information by our research staff. We hope that this information 
will help you. 



^-Howard A. Kinhart, The Effect of Supervision on High School English . 
Johns Hopkins University, 1941, p. 102. 

^N. L. Gage, Philip J. Runkel, and B. B. Chatterjee, Equilibrium Theory 
and Behavior Change; An Experiment in Feedback from Pupils to Teachers , 
(Urbana, Illinois? Bureau of Research, University of Illinois, 1960), 
p. 90. 

^Roy C. Bryan, Pupil Rating of Secondary School Teachers . Contributions 
to Education No. 708, (New York: Teachers College, Columbia University, 

1937), pp. 66-71. 

^Percival M. Symonds, "Characteristics of the Effective Teacher Based on 
Pupil Evaluations." Journal of Experimental Education , 23 (June 1955), 
pp. 289-310. 

5j. A. Starrak, "Student Rating of Instruction." Journal of Higher 
Education , 5 (February 1934), pp. 88-90. 
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b A. J. Druckers and Ho A. Reimners, "Do Alumni and Students Differ 
in Their Attitudes Toward Instructors?" Journal of Educational 
Psychology , 62 (March 1951) , p . 142 . 

^ Abraham S. Goodhartz, "Student Attitudes and Opinions Relating to 
Teaching at Brooklyn College, " School and Society . 68 
(November 20, 1948), pp. 345-9. 

8 H. H. Remmers, "The Relationship Between Students 1 Marks and 
Student Attitude Toward Instructors," School and Societ y. 28 
(December 15, 1928), pp. 759-60. 

^Earl Hudelson, "The Validity of Student Rating of Instructors," 

School and Society . 73 (April 28, 1951), pp. 265-66. 

10 H . H. Remmers, "Rating Methods in Research on Teaching," Handbook 
o f Research on Teaching , p. 367. 

1^-Arvil S. Barr, "The Measurement and Prediction of Teaching Efficiency- 
A Summary of Investigations." Journal of Experimental Education , 

(June 1948), p. 216. 

l^Arvil S. Barr, "The Measurement of Teacher Characteristics and 

Predictions of Teaching Efficiency." Review of Educational Researc h, 
22 (June 1952), p. 172. 

l^Quinn McNemar, "Opinion-Attitude Methodology." Psychological 
Bulletin , 43 (July 1946), p. 311. 

^Robert Bennett Hayes, "A Measure of Student Attitude Toward Teaching 
Effectiveness." (Unpublished D. Ed. dissertation. The, Pennsylvania 
State University, 1961), pp. 107-108. 

•^Robert B. Hayes, "A Way to Measure Classroom Teaching Effectiveness." 
Journal of Teacher Education , 14 (June 1963), pp. 168-176. 



If you have any questions or suggestions, please contact : 

Dr. Robert B. Hayes 
Supervisor of Research 
Bureau of Research 
Harrisburg , Pennsylvania 
Area Code 717 787-4860 
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February 14, 1966 

To selected Tee her s in Feedback Study 

Re: Pupil-Teacher Rating - Can It; Help You? 

We want to share some information with you which has been obtained 
in our feedback study and which may help you to understand and to use 
the results of ratings by your pupils. First, we want to emphasise 
that these results are strictly confidential; no one except you is 
being given pupil ratings concerning you as an individual. 

Next we want you to know that the results of this study confirm 
previous research findings that pupils are reliable raters. You will 
recall that the first nine items of the Hayes Pupil-Teacher Reaction 
Scale concern the teaching process. A correlation of .91 has been 
obtained between teacher ratings in October and three weeks later. 

This (test re-test) correlation of .91 is very clc *e to a perfect 
correlation of 1.00. The length of the average interval ef 21 days 
ui tween ratings and also the fact that 64 teachers and 1683 pupils 
were involved makes this high stability coefficient all the more 
remarkable. 

We also have some evidence of the validity of these ratings. 

You will recall that items 10 through 17 ranged from "This instructor 
is the very best I f ve ever had," to "This teacher is the worst I’ve 
ever had". The "very best" were given a score of seven, the "best" 
were given a score of six, etc., so that "The worst teachers" re- 
ceived a score of zero. The coefficient of correlation between such 
scores and scores for the first nine items on the Hayes Scale was .62. 

We hope that this evidence will increase your confidence in 
using the results reflected in your average ratings by your students. 

A few students can be expected to be much too far to one extreme or the 
other. The average for your class should be fairly reliable and suffi- 
ciently accurate to give you some clues as to how you might improve 
your teaching. 
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Since you may wish to compare yourself with the other teachers 
in this study, we are including the following tables: 

First Cycle - Student Rating Results for 64 Teachers 
Percent t y Item Alternatives 





a_ 


b 


c 


d 












Item 


1 


30 


45 


20 


5 


o 


49 


26 


22 


3 


3 


21 


49 


25 


5 


4 


11 


42 


46 


1 


5 


35 


51 


10 


4 


6 


56 


30 


13 


1 


7 


13 


35 


43 


9 


8 


28 


45 


18 


9 


9 


44 


47 


6 


3 



Second Cycle - Student Rating Results for 64 Teachers 
Percent by Item Alternatives 





a 


b 


jC 


d 












Item 


1 


25 


47 


24 


4 


2 


44 


30 


23 


3 


3 


20 


51 


24 


5 


4 


12 


46 


40 


2 


5 


32 


54 


10 


4 


6 


50 


34 


14 


2 


7 


16 


39 


38 


7 


8 


24 


47 


21 


6 


9 


43 


48 


6 


3 
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Third Cycle - Student Rating Results for 16 Teachers 
Percent by Item Alternatives 







b 


£ 


d 


Item 


- 








1 


21 


49 


26 


4 


2 


37 


33 


28 


2 


3 


20 


49 


26 


5 


4 


10 


51 


37 


2 


5 


33 


51 


10 


6 


6 


46 


33 


17 


4 


7 


13 


41 


39 


7 


8 


20 


47 


24 


9 


9 


38 


50 


6 


6 



Fourth Cycle - Student Rating Results for 16 Teachers 
Percent by Item Alternatives 





£ 


b 


c 


d 












Item 


1 


19 


50 


27 


4 


2 


36 


35 


25 


4 


3 


19 


51 


26 


A 


4 


11 


48 


38 


3 


5 


28 


54 


14 


4 


6 


41 


37 


19 


3 


7 


15 


40 


37 


8 


8 


20 


45 


25 


10 


9 


37 


51 


7 


5 



We hope that the above information plus the results already 
furnished you will provide guidance toward self-directed improvement 
in your classroom behavior. Teaching will be more truly a profession 
if reliable instructional principles are established which crn be 
practiced confidently in the classroom. 

Sincerely yours. 



Robert B, Hayes 
Director 

Cooperative Research 
Project 3010 



o 

ERIC 
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Interaction Analysis 
Feedback - Sheet 1 



Teacher_ 

School 

Date of Observation 



Your attention is invited to page 12 of the manual, "The Role 
of the Teacher in the Classroom^. During the period you were 
observe, the percent of time by interaction categories was as 
follows: 



Categories , % of Time 

1. Accepting student feeling 

2 . Praising 

3. Accepting or using student ideas " 

4. Asking questions for students to answer 

5. Lecturing 

6. Gx Hag directions 

7 . Criticizing students or justifying teacher authority 

8„ Talk by students in response to teacher ' 

9. Talk initiated by students ” 

10. Periods of silence or confusion 



There is some evidence to indicate that effective teachers have 
a large repertoire of behaviors. Some questions you might ask your- 
self are: 

Do you think there are any of the above categories which you 
should use more? 

Are there any that you might want to use tc a lesser extent? 

For your information, the total percent of teacher talk (categories 
1-7) was . 



The total percent of student talk was 



Do you feel the need to modify your teaching? 

There are no hard and fast rules. It is up to you to make the 
decisions. 



ERIC 
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Interaction Analysis 
Feedback - Sheet 2 



Teacher 



Are you typically a direct or indirect teacher? 

If over half the teacher talk is in the direct categories, you 
are more direct than indirect. A crude measure of teacher influence 
is the "big I/D ratio" which consists of dividing all tallies - column 
totals - in categories 1, 2, 3, and 4 by the tallies in 5, 6, and 7. 
Your I/D ratio was 



Certain occasions require direct behavior. Yet there is some 
evidence that students tend to learn more from the more indirect than 
from the more direct teachers. 

The "small i/d ratio," or revised i/d, is computed by deleting 
categories 4 (teacher questions) and 5 (lectures) from the big I/D 
ratio. The i/d ratio is less influenced by subject content and 
indicates the kind of emphasis given to motivation versus control. 
Your small or revised i/d ratio (categories 1, 2, and 3 divided by 6 
and 7) was . 



Do you spend enought time accepting or praising student ideas 
and feelings? 

The following shows the percent of time for each cell in this 
extended indirect influence area. 

Your Extended Indirect Influence 



1 


1 


2 


3 


2 








. 3 ! 









For example, a high percent of "2,2's" would mean a lot of extended 
statements of praise. An explanation of praise may give insight to 
students which they will find useful in guiding their future behavior. 

Past experience has indicated that Category 3 (Accepting Student 
Ideas) has a particularly important relationship to student achievement. 
Should you develop student ideas more thoroughly so that : hey feel you have 
really accepted their ideas? 
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Interaction Analysis 
Feedback - Sheet 3 

Teacher 

% 

Do students tend to resist you? influence? 

The following part of an interaction analysis matrix may be called 
the "vicicus circle". 



"Vicious Circle" 
6 7 

6 



7 



This name otiginated because it may reflect teacher directions 
followed by student resistance and then teacher criticism and more 
directions. A high percent in this area often indicates overt student 
resistance. The percent of time you spent in this area is indicated 
in the above portion of your matrix. 

What kind of student responses occur after teacher talk? 



8 9 




The above entries show the percent of time for each cell. A high 
percentage in column 8 compared to column 9 may indicate that your talk 
results in relatively predictable, memory-type student answers. More 
9's tend to indicate that students feel free to initiate talk and to 
expand on their own ideas. By examining the sequence of what preceded 
an 8 or 9, one can reconstruct the stimuli which triggered student 
participation. 

For example, the number entered in a Row 2, Column 9 cell indicates 
the percent of time teacher praise was followed by a student initiated 
statement. The 4,8 cell indicates the percent of time teacher questions 
were followed by predictable, memory-type student answers. 



APPENDIX E (CONTINUED) 
Interaction Analysis 
Feedback - Sheet 4 



Teacher 



What happens after the students talk? 
Do you praise or accept their ideas? 
Do you ‘tend to be critical? 



8 



























•! 

1 


1 

! l 



Vi LJ.LUC J.U L.11C U , ,2 ciU' 

you tend to accept or use student talk. 



Your I/D ratio for rows 8 and S is 



Your small i/d ratio for rows 8 and 9 is 



Attached is the matrix for the entire period that you were observed. 
By referring to the manual, "The Role of the Teacher in the Classroom", 
you may extract additional information from the enclosed matrix. 
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Interaction Analysis 
Feedback - Sheet 5 



Teacher 



COLUMNS 



1 Z 3 V f -6 7 8 9 /o Ktel 




NOTE: To refresh your memory as to how this natrix was recorded, see 

pages 25-28 of the manual, "The Role of the Teacher in the Classroom". 
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October 26, 1965 

To Teachers Selected to Receive Pre-Test Results 

You have beep selected to receive pre-test results in the feedback 
experiment. We hope that this information will be of help in under- 
standing your students and their needs for the particular class which 
Mr. Keim and Mr. Neiman have visited. 

One of the pre-tests was the Remmers Scale to measure attitude 
of students toward school subjects. Your attention is invited to the 
enclosed copy of this scale and to the manual, "The Purdue Master 
Attitude Scales." Your particular attention is invited to the last 
two pages of this manual which deal with scoring of the attitude scale 
and which will help you interpret the average scale score of your class 
which was . 

Also enclosed is a roster showing the intellige ice quotients and 
the percentiles for your students. These percentiles are based on your 
local tenth grade class. A percentile may be defined as that point 
below which the indicated percent of other cases (pupils) score in 
comparison with a particular case (pupil). 

Finally, we are enclosing a sheet for each of your students in 
the same class showing their standard scores and percentiles (based on 
your local tenth-grade class) on the following subtests of the Iowa 
Test of Educational Development: 

1. Understanding of Basic Social Studies 

2. General Background in the Natural Sciences 

3. Correctness and Appropriateness of Expression 

4. Ability to Do Quantitative Thinking 

5. Interpretation of Social Studies Reading Materials 

6. Interpretation of Natural Science Reading Materials 

7. Interpretation of Literature Materials 

8 . General Vocabulary 

9. Use of Sources of Information (Reference Books, Card 
Catalogues, Encyclopedias, etc.) 

The above numbers can be used to determine test areas indicated by 
the ITED sheets. 
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Your high school guidance counselor should be consulted for 
help in interpreting and, using the results. He (or she) can show 
you the following: 

1. A report of local percentile norms and a frequency 
distribution for the entire tenth grade class in 
your school 

2. Manual for the School Administrator 

3. Examiner's Manual 

4. Manual entitled "How to Use Test Results" 

We wish to thank you for your cooperation in this study. 

Sincerely, 



Robert B. Hayes 
Supervisor of Research 



RBH: j 
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Commonwealth of Pennsylvania 
Department of Public Instruction 
Box 911, Harrisburg, Pennsylvania 17126 

January 24, 1966 
To Selected Teachers in Feedback Experiment 

Re: Interaction Analysis - Can It Help You? 

The problems of improving teacher preparation and inservice training 
programs extend like a massive cold front. These phenomena are quite 
common in Pennsylvania. You might note that the work we have done on 
.interaction analysis so far has affected teacher preparation and inservice 
’training to the same extent that lighting a match affects Pennsylvania 
Weather. But we who come from the frozen north get a great deal of 
pleasure out of lighting a match and enjoy the warmth it sheds, however 
small . 

As VT e see it, the major problem is: how can a teacher obtain 

objective information about his clas&room behavior — information that is 
rich enough and clear enough to guidt his steps toward self-directed 
improvement? This is a question of feedback. The principle of feedback 
Is becoming more popular. It supports equally Thorndike's renaissance in 
the field of learning machines, provides the cutting edge of triad train- 
ing in human relations , and helps our spaceman keep a rendezvous with the 
moon. Lack of adequate feedback has plagued teacher training for cen- 
turies, and no doubt was the major gripe of the neophytes in the high ? 
priesthood of the Pharaoh's court. 

The principle of feedback is clear enough; its application in an 
engineering cense is quite another question. Just how can a teacher 
compare hi .3 performance with his own intentions under classroom conditions? 

Blueprints for providing feedback in human relations training can be 
found in a number of references . A reference that should not be overlooked 
is Festinger's (1954) "Theory of Social Comparison Processes." Let me 
remind you of some of its most relevant hypotheses. 

1. There exists, in the human organism, a drive to evaluate his 
opinions and abilities. 

2. To the extent that objective, non-social means are not available, 
people evaluate their opinions and abilities by comparing them 
with the opinions and abilities of others. 
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Teachers are just as interested in comparing their opinions and 
abilities as any other individuals. With regard to teaching, most 
comparisons are made without benefit of objective raethode. As a 
result, the comparisons are often abstract generalizations that have 
little relation to overt behavior, stereotypic value judgments are 
frequently expressed, and opinions are compared much more often than 
hard-to-def ine abilities, 

3. The tendency to compare oneself with some other specific 
person decreases as the difference between his opinion 
or ability and one’s own increases. 

Teachers are equally prone to check each other’s feathers before 
they flock together. The tendency to seek confirmation of current 
beliefs restricts innovation and the spirit of inquiry. Pressures 
toward uniformity that arise from such comparison processes present 
a formidable resistance to change,, 

4. When an objective, non-social basis for evaluating one’s 
opinion or ability is readily available, persons will not 
evaluate their opinions or abilities by comparison with 
others. 



We can all hope that this last hypothesis of Festinger’s is 
correct. It suggests that a more objective system of feedback will be 
preferred by teachers. Yet, the premise that interaction analysis is 
more objective than swapping opinions does not decrease the initial 
resistance of teachers whose skepticism has had years of nourishment. 

The problem of creating incentives in inservice training remains. 

It is the teacher’s job to decide whether this information is 
consistent or inconsistent with his own intentions. He must decide what 
changes, if any, are desirable. 

We have no research evidence to support the notion that an increase 
in student verbal participation per se improves learning. We do have 
research results to indicate that students learn more working with 
teachers who skillfully elicit, clarify, and challenge student ideas. 
Merely increasing student participation by asking questions is not 
enough. 



Interaction analysis serves two functions in this project. First, 
it is being used as a pre- and post-measure of success in helping 
teachers modify their o/ert behavior, in order to make it more consistent 
with their own self-improvement goals. Second, it is used as a working 
tool to feed back information during training. 
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In its present form, interaction analysis is cumbersome and 
expensive. It is also rather a crude instrument for such a delicate 
task; but as one teacher put it, "You don't need a razor to cut 
butter."! 



What is Interaction Analysis? 

For comparative purposes, it might be a fair analogy to say 
that Interaction Analysis is to teaching what the movie camera is to 
modern sports. That is, Interaction Analysis and a movie camera are 
both devices which enable their users to record accurately and pre- 
serve for future study the activities of a given past event. 

For example, a coaching staff and players are able to watch, 
stop, back up, review, and thus study films of a previous game's 
action. In this way they can zero in on each and any desired phase 
of the action. Or, a photo-finish of a horse race can accurately 
reveal the true winner, even in the case of a close finish where a 
winner cannot be determined immediately on the spot by the human eye. 

And now, with the advent of the "isolated camera", even the home- 
viewing T-V audience can witness a slow motion rerun of a crucial 
baseball or football play — just instants after it occurred and as 
a result, oe in an advantageous position to second-guess the umpire I 
But you might well ask what all of this has to do with Interaction 
Analysis and the teacher. 

Now, for the first time, in Interaction Analysis teachers have 
available a "camera" with which they can "focus in on" and "photograph" 
their teaching behavior. Just as the viewing of a past sporting event 
allows the viewer to analyze accurately the most detailed action of a 
given sequence on the playing field, so will the study of the Interaction 
Analysis data and matrix of a classroom interaction enable the teacher 
to assess his classroom teaching verbal behavior. Reviewing Interaction 
Analysis data and the final matrix allows the teacher to identify his 
weaknessess and mistakes. With proper revision and practice he is in 
a good polities to improve on his future teaching. 

What are the procedures of Interaction Analysis? Before de- 
scribing the mechanics of the system per se, several things should be 
made clear. First, Interaction Analysis is a descriptive system — 



lxhe foregoing two pages are quoted and adapted from a paper by 
Med A. Flanders, "Interaction Analysis: A Technique for Quantifying 

Teacher Influence." 
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not a diagnostic one. To go back to the analogy of the movie 
camera; the print exposed in the camera only reproduces unbiasedly 
what it 1 sees". The camera itself cannot point out the specific 
weaknessess or mistakes of the players. Through careful study of 
the film, however, professionals are able to analyze crucial "frames" 
and with careful thought make relevant inferences Then, with proper 
coaching and practice, the player (s) can revise his (their) behavior 
and, as i result become more skillful and proficient in the future. 
The same is true of the teacher. When the data from an Interaction 
Analysis observation is plotted in a matrix, the teacher has avail- 
able a "picture" of his teaching behavior. With careful study, the 
professional teacher can analyze his classroom behavior, identify 
his weaknesses and mistakes, and with proper revision and practice, 
improve on his future teaching effectiveness. 

Second, Interaction Analysis is an objective system. Previous 
to the use of movies in boxing, a boxer’s coaching staff had to plan 
their training and fighting strategy after hours of first-hand obser- 
vation of their man in action. Constantly they would discuss with 
each other what they saw (or thought they saw) him do. Needless to 
say, there couLlaually arose numerous disagreements between these 
individuals, since their observations were based on subjective 
judgments and thus open to inaccuracies. But when the same coaching 
staff takes advantage of th - * movie camera, they advance their judg- 
ments from the subjective level to the objective level and as a 
result their judgments become more accurate. Before Interaction 
Analysis the evaluation of teacher effectiveness was almost always 
limited to the dimensions of subjectivity. Administrators and 
supervisors personally visited the teacher’s classroom and made 
first-hand observations. They based their final evaluations on what 
they observed or thought they observed. Reports were based on such 
subjective criteria as the apparent "pleasantness" of the teacher, 
the observer’s personal opinion of the teacher’s skill in handling 
classroom situations, and the visible physical conditions of the 
classroom — lighting, ventilation, seating arrangement, displays, 
bulletin boards, and the like. Rarely if ever were their evaluations 
based upon any direct effect the tea ner’s behavior was having on 
the pupils, because observers had no means by which to assess this 
effect. With Interaction Analysis, the precise quality and sequence 
of the teacher's (and pupil's) verbal behavior can be recorded and 
used in future supervisory sessions. Not only is such data more 
objective, but often the resultant findings are a powerful 
revelation — to both observer and the teacher. 

Third, Interaction Analysis is a skill, and can be learned. 
Before an amateur photographer can reach a high level of proficiency 
in taking movies, he must gain considerable understanding of and 
practice with a movie camera and its operation. Much the same is 
true in the use of Interaction Analysis. Generally the average 
teacher can learn the basic skills involved in taking Interaction 
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Analysis in six to eight hours of study. However, it should be made 
clear that to gain proficiency and skill in the use and interpretation 
of the data requires more than the initial six or eight hours — perhaps 
more in the neighborhood of 30 or more hours. However, the point 
still remains the Interaction Analysis is a skill and can be learned 
by any classroom teacher if he is willing to spend the cime and effort 
necessary. 

Fourth, Interaction Analysis will serve and benefit a teacher 
only to the extent he is willing and able to help himself. That is, 
he must voluntarily want to put his teaching behavior before the critical 
eye of the system. For some teachers, the initial idea of having his 
classroom behavior laid open and analyzed may present a threat. But if 
the system is to benefit the teacher he must be willing to accept the 
challenge. Next, he should have an idea of what to look for in the data 
and how to interpret what he sees there * In other words, a teacher must 
have at his disposal some previous knowledge concerning the teaching- 
learning act. His own mind and thinking must be clear as to what his 
personal role is as a ’’facilitator" of learning. He may secure this 
image by reviewing his sociology, physiology, human relations, individual 
behavior, subject content, and the nature of his own person. When a 
teacher clearly understands his role as a ’’facilitator" of learning, he 
can plan a lesson, teach it while Interaction Analysis data is obtained, 
and afterward analyze the data to determine if he, in fact, successfully 
performed in the classroom in the way he had planned. Where there are 
discrepancies or weakness, he can provide for correction and strength- 
ening in the future. In this way, a teacher is able to "get the pulse" 
of his classroom behavior and thereby be in a position to plan for 
improvement . 



The Procedure of Interaction Analysis 

What does a person do when he takes Interaction Analysis? What is 
involved? As stated above. Interaction Analysis is a skill and the 
simple mechanics of it can be learned in a rather short period of time — 
six to eight hours. The process iteself is discussed below. 

The seven categories assigned to teacher talk are divided into two 
types of teacher influence — indirect and direct. Categories number 
one through four represent indirect teacher influence and categories 
number five through seven represent direct teacher influence. 

Indirect influence stimulates student participation, resulting in 
student freedom (acceptance of student feelings, praise and encourage- 
ment, acceptance and clarification of student’s ideas, and solicitation 
of student participation through questioning) . On the other hand direct 
influence (lecturing, giving commands, rejecting student behavior and 
justification of teacher role) shifts the participatory emphasis from 
the student to the teacher. As a result, student freedom is limited and 
restricted while student compliance to the teacher’s demand is increased. 
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Division of student, talk into two categories permits a proper 
distinction to be drawn between teacher-initiated student talk and 
student -initiated student talk. For example, suppose the teacher 
should ask the question "what city is the capitol of Nevada?" The 
student's answer to the question would be a result of the teacher's 
question which called for a single specific correct answer. This 
student talk would be teacher-initiated (category eight) . But 
suppose the same student, or another member of the class, should 
voluntarily respond to the above episode by relating certain 
interesting events connected with a trip he took with his family 
through Nevada. This would be recorded as student -initiated talk 
(category nine) since the student responded on his own accord and 
initiative. By making a simple mathematical comparison of teacher- 
ini txafed and student— initiated student talk much can be learned 
about the personal nature and classroom behavior of a teacher. 

For a more detailed discussion of the procedure of Interaction 
Analysis and how to develop the skill , refer to The Role of the 
Teacher in the Classroom * A Manual for Understan ding and Improving 
Ti ...cher's Classroom Behavior (1). ~ 

An operational knowledge of Interaction Analysis can be of 
significant value in at least three different ways. First, it is a 
valuable supervisory tool — both at the pre-service (college) level 
and at the inservice level. Using Interaction Analysis, supervisors 
are able to assess accurately and objectively the verbal behavior 
of the teachers under his supervision. With proper advice and counsel 
the supervisor can help the individual teacher to improve his class- 
room behavior and ultimately make his teaching more effective. 

Second, individual inservice teachers can use Interaction 
Analysis in analyzing their own verbal behavior. They can record a 
class period, later take Interaction Analysis from the tape and 
subsequently analyze and study the data for future improvement. Some 
school systems have found success in encouraging groups of two or 
more teachers to work together as teams in recording and analyzing 
each others classroom verbal interaction. 

Third, an understanding of Interaction Analysis serves as an 
objective framework around which pre-service education courses can 
be built. The categorical system helps students to identify 
specific teaching behaviors and behavioral sequences «. 

The observer situates himself in the classroom in a comfortable 
position where he can clearly hear and view the participants. At 
the end of each three second interval, he decides which of the 10 
categories best describes the verbal behavior just completed. He 
writes this number down on a tally sheet, usually arranged in 
vertical columns of approximately 20-25 tallies, each column 
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representing one minute of classroom interaction. Simultaneously 
with the recording of the pi ious three-second period of inter- 
action, the observer assesses the present three-second period. He 
continues this same sequential pattern over and over at as steady 
a tempo as possible. The completed series of category numbers 
represents a sequential record of the verbal interaction that 
occurred in the class period just observed. If he wishes, the 
observer may occasionally jot dow i brief marginal notes which are 
related to a particular segment 01 the observational data. Any 
major change in class formation, ccmnunicat ion pattern, or subject 
under consideration is identified by drawing a double line and 
recording the clock time* Upon completion of the observational data 
(usually no less than 20 minutes) the observer takes leave to a 
nearby room and completes his report by giving : general description 
of each separate activity period indicated by the double lines in- 
cluding the nature of the activity, the class formation and the 
teacher r s position. Any other notes or facts which might be 
necessary for an adequate interpretation or the total observational 
period are also included. 

Flanders System of Interaction Analysis consists of ten 
categories — seven assigned to teacher talk, two assigned to student 
talk, and a single category to designate short periods of "silence" 
or "confusion". A detailed outline of the category system is 
included on the next page. 
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SUMMARY OF 

CATEGORIES FOR INTERACTION ANALYSIS 



TEACHER 



TALK 



INDIRECT 



INFLUENCE 



DIRECT 



INFLUENCE 



1 . 



2 . 



3 . 



4 . 



5 . 



6 . 



7 . 



STUDENT 



TALK 



d. 



9 . 



10 . 



* ACCEPTS FEELING: accepts and clarifies 

the feeling tone of the students in a 
non threatening manner. Feelings may be 
positive or negative. Predicting and 
recalling feelings are included. 

*PRAISES OR ENCOURAGES : praises or en- 

courages student action or behavior. 

Jokes that release tension, not at the 
expense of another individual, nodding 
head or saying "uhhuh?" or "go on" are 
included. 

*ACCEPTS OR USES IDEAS OF STUDENT: 
clarifying, building or developing ideas 
or suggestions by a student. As teacher 
brings more of his own ideas into play, 
shift to category five. 

*ASKS QUESTONS: asking a question about 

content or procedure with the intent that 
a student answer. 

*LECTURES : giving facts or opinions about 

content or procedure; expressing his own 
idea; asking rhetorical questions. 

*GIVES DIRECTIONS: directions, commands or 

orders witt which a student is expected 
to comply. 

^CRITICIZES OR JUSTIFIES AUTHORITY: 
statements intended to change student 
behavior from nonacceptable to acceptable 
pattern; bawling someone out; stating why 
the teacher is doing what he is doing, 
extreme self- reference. 



*STUDENT TALK-RESPONSE: talk by students 

in response to teacher. Teacher initiates 
the contact or solicits student statement. 

*STUDENT TALK-INITIATION: talk by students, 

which they initiate. If "calling on" 
student is only to indicate who may talk 
next, observer must decide whether student 
wanted to talk. If he did, use this category. 



*SILENCE OR CONFUSION: pauses, short periods 

of silence and periods of confusion in which 
communication cannot be understood by the 
observer. 
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Employing this notion in small-group role-playing situations does 
a lot to bring educational theory to a practical level. In addition, 
pre-service teachers who have an operational knowledge of Interaction 
Analysis can analyze different styles of teaching. Thus, they can 
make objective judgments as to whether or not these styles could be 
used effectively in their own personal teaching methods and techniques. 
This allows the teacher free choice as well as objective judgment in 
formulating his own teaching repertoire. 



Interaction Analysis in In-service Education 

1. In supervision . The notion that pre-service education 
is not an end, but rather a beginning is too often overlooked. As a 
result we fail miserably when we neglect in-service programs of super- 
vision and guidance — especially in the cases of many beginning 
teachers. Consequently they are allowed (more often forced ) to shift 
for themselves and ultimately meet with failures in their teaching. 
Interaction Analysis provides one ray of hope in this otherwise dismal 
picture. As in pre-service education, discussed above. Interaction 
Analysis can be used as an effective means of objectively assessing 
the in-service teacher's classroom verbal behavior. This, along with 
careful thought , planning and experimentation can do much to improve 
teaching effectiveness. 

2. As a team approach . As mentioned earlier, some admin- 
istrators have recently met with success by encouraging their teachers 
to analyze and experiment with their teaching behavior through a team 
approach. Such an approach involves two or more teachers who are 
granted sufficient free time to observe each using Interaction Analysis. 
In this way they are better enabled to get at both individual as well 

as common instructional problems. 

3. As a self-analysis approach . Perhaps the most unique 
feature of Interaction Analysis is the fact that an individual teacher 
can use it even in the absence of outside assistance. He simply 
records a live class session on a tape recorder. Later on, at his 
convenience, he plays it back, takes Interaction Analysis on his 
lesson, and plots the data in a matrix. Because of this feature, a 
single teacher has the distinct advantage of being able to analyze 
his own classroom verbal behavior, even when other assistance is 
unavailable. 



Interaction Analysis as a Research Tool 

1. In the analysis , classification , and study of teacher 
types . Traditionally educationists have been noted for their 
reluctance to define effectiveness in teaching. A major reason for 
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this hesitancy is that much in teacher evaluation is of a subjective 
nature. With Interaction Analysis , however, we are in a position to 
analyze objectively the verbal dimension of instruction. Hopefully 
the future will witness significant advances in the assessment and 
identification of various types of teacher verbal behavior. 

2 . In studying the effects of teacher types on learning . 
Suppose we perfect our ability to identify accurately different types 
of teacher verbal behavior. By controlling this important variable 
in an experimental design, we put ourselves in a position to measure 
the effect that a given teacher-type will have on a variety of 
learning factors including student perception, motivation, achieve- 
ment, etc. 



3 . Measuring change in teacher behavior . Many things 
that are done in teacher education programs are done with a hope and 
a prayer. That is, we include in teacher preparation programs 
certain concepts, methods, and content because it seems "logical" 
that these things will result in changing the teaching behavior of 
the teacher. One reason we do this is that the evaluation practices 
employed in teacher education are of a highly subjective nature. 
However, if we can objectively define and assess the various pro- 
gressive levels of teacher behavior, we can then establish certain 
definite guidelines and bench marks. Using these we can begin to 
measure teacher growth and subsequently the effectiveness of teacher 
education programs. Interaction Analysis is one of the first such 
objective evaluation devices to come upon the educational scene. 



In Conclusion 



It is the opinion of this writer that Interaction Analysis 
has made (will make) no less than two contributions which will affect 
significantly the future of the teaching profession. First, it serves 
as a monumental example of what must be done in the domain of 
objective teacher evaluation. This in itself should provide stimu- 
lation as well as a model for considerably more activity in this most 
vital phase of teaching. 

Second, with the creation of new and better methods and 
techniques for measuring teacher effectiveness, the future should 
witness a dramatic increase in the identification of reliable instruc- 
tional principles. These principles will be reliable because they 
will have been successfully tested by instruments which objective and 
therefore reliable themselves. With a multitude of tested and reli- 
able instructional principles from which to choose we will be able to 
formulate instructional programs which can be put into practice in 
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the classroom with confidence. When all of this becomes a reality, 
teaching will be one big step closer to being truly a profession!** 



Sincerely yours, 



Robert B. Hayes 
Director 

Cooperative Research Project 3010 



^The preceding eight pages were extracted from an article by 
Richard L. Ober, "A Review of Flanders System of Verbal Interaction 
Analysis." 
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STATISTICAL DATA AND ANALYSIS 
TABLE 1 

CLASS MEANS FOR INTELLIGENCE 







TREATMENTS 




















Means for 


Subjects 


1 


2 


3 


4 


5 


Total 


English 


101.87 


112.63 


108.61 


121.60 


103.12 




113.16 


115.04 


122.03 


123.83 


103.00 






11C. 33 


129.25 


112.45 


113.20 


95.86 






94.96 


112.27 


115.07 


92.78 


109.25 




Means 


105.58 


117.30 


114.54 


112.85 


102.81 


110c 62 


Math 


111.29 


114.40 


124.68 


115.36 


110.05 






121.05 


124.58 


116.28 


113.39 


107.44 






95.69 


111.95 


118.27 


115.62 


124.82 






121.96 


116.54 


108.61 


116.60 


112.40 




Means 


112.50 


1.16.87 


116.96 


115.24 


113.68 


115.05 



Science 


108.61 

111.17 

122.39 

116.21 


99.92 

121.07 

100.68 

103.62 


115.04 

104.63 

95.91 

116.57 


114.59 

120.89 

110.52 

93.55 


93.76 

118.83 

98.57 

100.59 




Means 


114.60 


106.32 


108.04 


109.89 


102.94 


108.36 


Social 


108.61 


113.12 


91.79 


120.75 


113.86 




Studies 


116.54 

110.52 

109.60 


129.04 

117.75 

105.10 


109.92 

123.76 

113.61 


115.04 

95.93 

124.94 


94.66 

116.35 

95.96 




# 

Means 


111.32 


116.25 


109.77 


114.16 


105.21 


111.34 


Means 111 • 00 

for Treatments 


114.19 


112.33 


113.04 


106.16 


111.34 
(Grand Mean) 
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TABLE 2 

ANALYSIS OF VARIANCE OF INTELLIGENCE 
ALL TREATMENTS (80 CLASSROOMS) 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


7006.783 






Treatments 


4 


622.767 


154.572 


i. 75 


Subjects 


3 


463.717 


155.692 


1.77 


Interaction 


. 12 


632. SOI 


52.7U8 


.60 


Within (error) 


60 


5287.798 


88.130 


- 



TABLE 3 

ANALYSIS OF VARIANCE OF INTELLIGENCE 
SUB-TREATMENTS (48 CLASSROOMS) 


Source of 




Sum of 


Variance 


F 


4 


Variation 


df 


Squares 


Estimate 


Ratio 




'’"otal 


47 


3551.868 








Sub-treatments 


5 


157.897 


31.579 


.38 




Subjects 


3 


201.240 


67.080 


.81 




Interaction 


15 


1212.044 


80.803 


.98 




Within (error) 


24 


1980 -687 


82.529 
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TABLE 4 

PRE-ITED CLASS ACHIEVEMENT MEANS 



TREATMENTS 



Subjects 


1 


2 


3 


4 


5 


Means for 


English 


12 . 80 


15.60 


14.96 


17.78 


13.45 


Total 




16.05 


16.44 


19 . 53 


21.20 


13.47 






16.10 


21.68 


16.23 


16.46 


11.68 






10.41 


17.40 


17.53 


11,01 


15.88 




Means 


13.84 


17.78 


17.06 


16.61 


13.62 


15.78 



Math 


16.57 

19.11 

10.85 

20.07 


16.20 

20.50 

15 . 50 
14.85 


20.57 

16.94 

19.53 

13.64 


16.86 

15.13 

16.50 

20.12 


14.53 

14.28 

22.32 

16.96 




Means 


16.65 


16.76 


17.67 


17.15 


17.02 


17.05 



Science 


16.66 


13.16 


17.25 


18.83 


11.50 






18.35 


22.23 


13.50 


20.79 


20.54 






21.20 


12.98 


15.32 


17.02 


10.65 






19.86 


14.12 


20.64 


V 97 


12.95 




Means 


19.02 


15.62 


16.68 


17.15 


13.91 


16.48 


Social 


16.13 


16.40 


11.47 


18.72 


17.83 




Studies 


15.77 


21.87 


16.25 


16.71 


11.90 






17.23 


17.36 


18.56 


12.06 


19.90 






14.07 


14.19 


15.25 


20.46 


10 . 75 




Means 


15.80 


17.46 


15.38 


16.99 


15.10 


16.15 



Means for 












Treatments 16 . 33 


16.90 


16.70 


16.98 


14.91 


16.36 



(Grand Mean) 
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TABLE 5 

ANALYSIS OF VARIANCE OF PRE-ITEu CLASS ACHIEVEMENT MEANS 
ALL TREATMENTS (80 CLASSROOMS) 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


811.042 






Treatments 


4 


46.227 


11.557 


1.05 


Subjects 


3 


17.423 


5.808 


.53 


Interaction 


12 


89.391 


7.449 


.68 


Within (error) 


60 


658.001 


10.967 





ANALYSIS 


TABLE 6 

OF VARIANCE OF P0ST-ITED ACHIEVEMENT MEANS 
SUB-TREATMENTS (48 CLASSROOMS) 


Source of 




Sum of 


Variance 


F 


Variation 


df 


Squares 


Estimate 


Ratio 


Total 


47 


396.878 






Treatments 


5 


18.158 


3.63 


.38 


Sub j ects 


3 


8.640 


2.88 


.30 


Interaction 


15 


141.455 


9.43 


.99 


Within (error) 


24 


228.625 


9.53 
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TABLE 7 

FOST-ITED CLASS ACHIEVEMENT MEANS 



TREATMENTS 



Subjects 


1 


2 


3 


4 


5 


Means for 


English 


14.08 


17.81 


16.57 


19.78 


14.97 


Total 




17.43 


16.92 


21.58 


23.64 


14.92 






17.94 


24.15 


18.70 


18.69 


13.38 






12.73 


18.24 


19.26 


12.01 


17.56 




Means 


15.55 


19.28 


19.03 


18.53 


15.21 


17.52 



Math 


17.93 

21.53 

13.15 

22.48 


18.35 

24.42 

17.95 

16.00 


21.43 

18.39 

21.47 

17.94 


18.23 

15.87 

17.38 

22.36 


16.16 

17.36 

22.41 

18.56 




Means 


18.77 


19.18 


19.81 


18.46 


18.62 


18.97 



Science 


18.93 


15.43 


20.36 


19.09 


13.56 






20.84 


23.02 


14.93 


22.48 


22.34 






22.49 


15.52 


17.32 


18.45 


12.07 






21.23 


15.56 


23.79 


13.22 


14.31 




Means 


20.87 


17.38 


19.10 


18.31 


15.57 


18.25 


Social 


18.09 


17.83 


12.68 


19.82 


18.57 




Studies 


16.93 


23.81 


17.06 


18.97 


12.99 






17.32 


18.93 


18.98 


14.04 


21.34 






17.53 


16.32 


15.73 


23.20 


12.18 




Means 


17.47 


19.22 


16.11 


19.01 


16.27 


17.62 



Means for 












Treatments 18.16 


18.77 


18.51 


18.58 


16.42 


18.09 



(Grand Mean) 
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TABLE 8 

ANALYSIS OF VARIANCE OF POST-ITED CLASS ACHIEVEMENT MEANS 

ALL TREATMENTS (80 CLASSROOMS) 



Source of 
Variation 


df 


Sum of - 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


839.361 






Treatments 


4 


58.801 


14.700 


1.36 


Subjects 


3 


26.965 


8.988 


.83 


Interaction 


12. 


105.014 


8.751 


.81 


Within (error) 


60 


648.581 


10.810 





TABLE 9 

ANALYSIS OF VARIANCE OF POST-ITED CLASS ACHIEVEMENT MEANS 

SUB-TREATMENTS (48 CLASSROOMS) 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


47 


420.775 


. 




Treatments 


5 


23.373 


4.675 


.48 


Subjects 


3 


24.701 


8.234 


.84 


Interaction 


15 


137.821 


9.188 


.94 


Within (error) 


24 


234.880 


9.787 
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TABLE 10 

CLASS MEANS FROM REMMERS SCALE TO MEASURE ANY SCHOOL SUBJECT 

PRE-TEST 



TREATMENTS 



Subjects 


1 


2 


3 


4 


5 


Means For 














Total 


English 


6.87 


7.24 


8.28 


7.24 


6.89 






8.32 


6.62 


7.88 


7.78 


6.62 






6.21 


7.62 


7.95 


6.77 


5.82 






6.15 


7.75 


7.30 


6.56 


7.43 




Mean 


6.89 


7.31 


7.85 


7.09 


6.69 


7.17 



Math 


5.76 


6.61 


7.31 


7.30 


7.70 






8.12 


6.42 


7.65 


8.22 


7.42 






7.68 


7.48 


8.30 


7.35 


7.89 






6.77 


5.91 


7.63 


7.63 


7.71 




Mean 


7.08 


6.61 


7.72 


7.63 


7.68 


7.34 



Science 


7.78 


6.96 


7.17 


7.64 


7.64 






6.57 


6.88 


6.38 


7.31 


7.45 






7.22 


6.60 


7.71 


8.19 


6.87 






7.37 


7.45 


6.89 


7.62 


6.59 




Mean 


7.24 


6.97 


7.04 


7.69 


7.14 


7.21 



Social 


8.05 


7.90 


5.74 


7.20 


6.01 




Studies 


6.21 


6.71 


6.73 


7.10 


4.62 






7.19 


5.66 


6.59 


7.29 


6.85 






7.01 


7.64 


6.07 


6.27 


6.59 




Mean 


7.12 


6.98 


6.28 


6.97 


6.02 


6.67 



Mean 












for Total 7.08 


6.97 


7.22 


7.34 


6.88 


' 7.10 



(Grand Mean) 
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TABLE 11 



CLASS 


MEANS 


FROM REMMERS 


SCALE 


TO MEASURE 


ANY SCHOOL SUBJECT 








POST- 


-TEST 












TREATMENTS 


















Means for 


Subjects 


1 


2 


3 


4 


5 


Total 


English 


7.47 


8.39 


8.27 


7.57 


7.11 






8.28 


7.06 


7.88 


7.76 


6.75 






7.20 


7.64 


8.04 


6.71 


6.07 






6.94 


6.91 


6.87 


6.68 


6.92 




Mean 


7.47 


7,50 


7.76 


7.18 


6.71 


7.33 



Math 


7.31 


8.01 


6.99 


7.24 


7.46 






8.01 


8.41 


7.45 


7.70 


6 . 76 






6.72 


8.14 


8.40 


7.43 


6.87 






7.58 


7.21 


6.85 


7.43 


7,84 




Mean 


7.41 


7.94 


7.42 


7.45 


7.23 


7.49 



Science 


8.27 


8.31 


7.40 


8.05 


7.07 






4.93 


8.13 


7.07 


8.20 


7.75 






7.89 


6.98 


8.13 


7.55 


6.65 






6.60 


8.32 


7.82 


7.99 


7.68 




Mean 


6.92 


7.93 


7.61 


7.95 


7.29 


7.54 



Social 


8.18 


8.16 


6.68 


8.05 


8.34 




Studies 


6.94 


7.91 


6.95 


7.68 


5.17 






8.02 


4.65 


7.42 


7.33 


7.33 






7.68 


7.62 


7.64 


7.41 


7.91 




Mean 


7.71 


7.09 


7.17 


7.62 


7.19 


7.35 



Means for 












Treatment 7 . 38 


7.62 


7.49 


7.55 


7.11 


7.43 
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i 



TABLE 12 

STUDENT ATTITUDES TOWARD SCHOOL SUBJECTS (PRE-TEST) 
BY TREATMENTS AND SUBJECTS 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


41.85 


_ 




Treatments 


4 


2.24 


.56 


1.37 


Subjects 


3. 


5.19 


1.73 


4.23** 


Interaction 


12 


9.87 


.823 


2.01* 


Within (error) 


60 


24.55 


.409 





* Significant at the .01 level 
** Significant at the .05 level 



TABLE 13 

STUDENT ATTITUDE TOWARD SCHOOL SUBJECTS (POST-TEST) 
BY TREATMENTS AND SUBJECTS 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


40.42 






Treatments 


4 


2.58 


.645 


1.22 


Subjects 


3 


.65 


.217 


.411 


Interaction 


12 


5.52 


.460 


.871 


Within (error) 


60 


31.67 


.528 
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TABLE 14 

STUDENT ATTITUDE TOWART SCHOOL SUBJECTS (PRE-TEST) 
BY SUB-TREATMENTS AND SUBJECTS 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


47 


24.94 






Sub- treatments 


5 


.95 


.190 


.349 


Sub j acts 


3 


1.90 


.633 


1.165 


Interaction 


15 ' 


9.05 


.603 


1.110 


Within (error) 


24 


13.04 


.543 


— 



STUDENT 


TABLE 15 

ATTITUDE TOWARD SCHOOL SUBJECTS (POST-TEST) 
BY SUB-TREATMENTS AND SUBJECTS 


Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


47 


29.45 






Sub-treatments 


5 


3.86 


.772 


1.063 


Sub j ects 


3 


.48 


.160 


.220 


Interaction 


15 


7.68 


.512 


.705 


Within (error) 


24 


17.43 


.726 
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TABLE 16 

STUDENT ATTITUDE TOWARD SCHOOL SUBJECTS 
ANALYSIS OF COVARIANCE (MULTIPLE CLASSIFICATION) 



TEST OF SIGNIFICANCE FOR TREATMENTS 



Source of 
Variation 




df 


Sum of Squares 
of Residuals 


Variance 

Estimate 


F 

Ratio 


Treatments 


+ Within 


63 


20.11 






Within 


• 


59 


18.28 


.31 


- 


Treatments 




4 


1.83 


.46 


1.48 




TEST OF 


SIGNIFICANCE FOR SUBJECTS 







Source of 
Variation 


df 


Sum of Squares 
of Residuals 


Variance 

Estimate 


F 

Ratio 


Subjects + Within 


62 


19.92 






Within (alone) 


59 


18.28 


.31 


- 


Subjects 


3 


1.64 


.55 


1.77 


TEST OF 


SIGNIFICANCE FOR INTERACTION 





Source of 
Variation 


df 


Sum of Squares 
o c Residuals 


Variance 

Estimate 


F 

Ratio 


Interaction 4- Within 


71 


26.59 






Within (alone) 


59 


18.28 


.31 


x 


Interaction 


12 


8.31 


.69 


2.24 * 



*Signif leant at the .05 level. 
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APPENDIX H (CONTINUED) 



TABLE 17 

MEANS FOR STUDENT-TEACHER RATINGS 
FIRST AND LAST CYCLE 



TREATMENTS 








TREATMENTS 






1 2 


3 


4 




1 


2 


3 


4 




Subjects — 


- English 




Means 






English 




Means 








for 










for 




Pre 




Total 






Post 




Total 


9 9 


7 


8 




7 


9 


7 


5 




1 7 


7 


9 




9 


7 


9 


3 




8 8 


7 


5 




9 


7 


4 


2 




6 6 


8 


7 




4 


4 


9 


7 





7.5 


7.5 


7.25 


7.25 


6.94 


7.25 


6.75 


7.25 


5.75 


6.94 






Math 










Math 






3 


8 


9 


9 




2 


9 


9 


9 




6 


7 


8 


8 




6 


8 


8 


9 




9 


9 


6 


9 




9 


9 


1 


9 




8 


9 


7 


8 




7 


9 


6 


7 




6.5 


8.25 


7.5 


8.5 


7.81 


6.0 


8.75 


6.0 


8.5 


7.44 






Science 








Science 




8 


7 


7 

r 


7 




9 


6 


8 


8 




8 


9 


8 


7 




9 


9 


8 


9 




5 


9 


5 


8 




5 


9 


7 


r 




1 


8 


8 


9 




2 


9 


5 


9 




5.5 


8.25 


7.0 


7.75 


7.44 


6.25 


8.25 


7.0 


8.25 


6.88 






Social 


. Studies 






Social 


Studies 


9 


6 


9 


9 




9 


1 


8 


9 




8 


9 


7 


9 




9 


9 


7 


9 




7 


5 


7 


9 




6 


5 


5 


8 




9 


9 


9 


7 




9 


9 


9 


7 




8.25 


7.25 


8.0 


8.5 


8.0 


8.25 


6.0 


7.25 


8.25 


7.69 


Means 


for Treatment 
















7.38 


6.65 


7.69 


7.55 


7.55 


7.13 


7.55 


8.0 


7.40 


7.29 



APPENDIX H (CONTINUED) 

TABLE 18 

ANALYSIS OF VARIANCE FOR HAYES REACTION SCALE 

CYCLE 1 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


153.86 


__ 


mmwmm 


Treatments 


3 


10.55 


3.52 


1.93 


Subjects 


3 


6.92 


2.31 


1.27 


Interaction 


9 


20.14 


2.24 


1.23 


Within (error) 


64 


116.25 


1.82 


— — 



ANALYSIS 


TABLE 19 

OF VARIANCE FOR HAYES 
CYCLE 6 


REACTION SCALE 




Source of 




Sum of 


Variance 


F 


Variation 


df 


Squares 


Estimate 


Ratio 


Total 


79 


372 . 39 


__ 


MM 


Treatments 


4 


8.33 


2.08 


.421 


Subjects 


3 


11.14 


3.71 


.753 


Interaction 


12 


57.17 


4.76 


.966 


Within (error) 


60 


295.75 


4.93 
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TABLE 20 

ANALYSIS OF VARIANCE FOR HAYES REACTION SCALE 
CYCLE 1 SUB-TRFATMENTS 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


47 


129.48 






Sub-treatments 


5 


16.86 


3.37 


1.09 


Subjects 


3 


5.06 


1.69 


.545 


Interaction 


15 


33.06 


2.20 


.709 


Within (error) 


24 


74.50 


3.10 





ANALYSIS 


TABLE 21 

OF VARIANCE FOR HAYES REACTION SCALE 
CYCLE 6 SUB-TREATMENTS 




Source of 
Variation 


df 




Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


47 




257.67 






Sub- treatments 


5 




47.42 


9.48 


1.82 


Subjects 


3 




.50 


.17 


.033 


Interaction 


15 




84.75 


5.65 


1.08 


Within (error) 


24 




1.25 


5.21 
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TABLE 22 

COEFFICIENTS OF REPRODUCIBILITY 
HAYES PUPIL-TEACHER REACTION SCALE 



Item 


First Rating 


Second Rating 


1 


.89 


.88 


2 


.79 


.86 


3 


.84 


.89 


4 


.77 


.81 


5 


.86 


.89 


6 


.80 


.81 


7 


.83 


.86 


8 


.85 


.88 


9 


.87 


.88 


Average 


.83 


.86 



TABLE 23 
MARGINALS 

HAYES PUPIL-TEACHER REACTION SCALE 



Item 


First Rating 


Second Rating 


1 


.76 


.72 


2 


.49 


.43 


3 


.70 


.72 


4 


.53 


.64 


5 • 


.35 


.33 


6 


.56 


.50 


7 


.49 


.54 


8 


.27 


.24 


9 


.34 


.39 


Average 


JP 


.50 



APPENDIX H’ (CONTINUED) 

TABLE 24 

INTENSITY ANALYSIS 

FOR CYCLES 1 AND 2 TREATMENTS 1, 2, 3 AND 4 

HAYES PUPIL-TEACHER REACTION SCALE 
CONTENT SCORE 



Intensity 

Score 


0 


1 


2 


3 


• 

4 


5 


6 


7 


8 


9 


Total 


Cum 

% tile 


9 


7 


0 


1 


0 


0 


0 


0 


0 


0 


70 


78 


100 


8 


4 


0 


1 


1 


0 


0 


1 


0 


55 


46 


108 


98 


7 


L. 


2 


2 


2 


0 


0 


3 


35 


80 


41 


170 


95 


6 


Z_ 


2 


5. 


2 


3 


5 


15 


59 


82 


51 


231 


91 


5 


15 


_4_ 


7 


3 


a. 


20 


48 


108 


52 


37 


302 


85 


4 




_S_ 


a 


LL_ 


__31_ 


-76- 


127 


143 


16 


24 


472 


77 


3 


-37_. 


21 . 


_2U 


48 


_1Q3 


151 


170 


67 


0 


5 


630 


65 


2 


52- 




-78 


.110 


-175 


155 


84 


1 


0 


__ 0 


703 


48 


1 




85 


-136 


—178. 


135 


65 


2 


0 


0 


0 


665 


30 


0 


91 


L38 


129 


92 


34 


0 


0 


0 


0 


0 


_484_ 


13 


Totals 


306 


316 


387 


447 


493 


472 


459 


413 


285 


274 


3843 


Cum 

% tiles 


08 


16 


26 


38 


51 


63 


_Z1 


85 


93 


100 






Midpoint 

of 

Content 
% tiles 


4 


12 


21 


32 


44.5 


57 


69 


80 


89 


96.5 






Median of 
Intensity 
X tiles 


29.5 


17 


21.1 


25,5 


38 


49.8 


61.9 


76.6 


90.5 


92.9 
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TABLE 25 

FOR CYCLE 6 TREATMENTS 1, 2, 3 and 4 
HAYES PUPIL-TEACHER REACTION SCALE 
CONTENT SCORE 



Intensity 

Score 


0 


1 


2 


3 


4 


5 


6 


7 


8 


9 


Total 


Cum 

% tile 


9 


7 


0 


1 


0 


C 


0 


0 


0 


0 


72 


80 


100 


8 


3 


1 


5 


0 


0 


0 


0 


0 


14 


42 


65 


96 


7 

6 


12 


2 


1 


0 


o 


o 


1 


4 


29 


43 


92 


93 


13 


7 


3 


0 


3 


1 


2 


8 


42 


54 


133 


89 


5 

4 


18 


10 


8 


2 


0 


2 


21 


32 


61 


34 


188 


82 


18 


14 


3 


8 


6 


9 


16 


49 


44 


23 


190 


73 


.1 


24 


20 


12 


20 


30 


35 


58 


51 


33 


6 


289 


64 


2 


31 


31 


21 


34 


51 


49 


70 


38 


11 


0 


336 


51 


1 


33 


34 


62 


57 


55 


80 


38 


9 


0 


0 


368 


35 


0 


30 


62 


60 


81 


69 


42 


13 


0 


0 


0 


357 


17 


Totals 


189 


_ 181 


176 


202 


214 


218 


219 


A9± 


234 


274 


2098 


Cum 

% tiles 


9 


,18 


26 


36 


46 


56 


67 


76 


87 


100 




Midpoint ■ 
of 

Content 
% tiles 


4.5 


13.5 _ 


22_ 


31 


41 


51_ 


61.5 


71.5 


81.5 


93.5 


Median of 
Intensity 
% tiles 


51.3 


32.1 


25.13 


23.3 


29.4 


32.1 


f>8.4 


63.4 


77.3 


90.9 
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TABLE 26 

COEFFICIENTS OF CONSISTENCY 
HAYES PUPIL-TEACHER REACTION SCALE 

PUPILS IN TREATMENT 1 



Cycles 


r 


N 


Number of Weeks 
Between Ratines 


1 & 2 


.75 


333 


3 


2 & 3 


.80 


429 


3 


3 & n 


.85 


423 


3 


1 & 3 


.72 


433 


6 


2 & 4 


.77 


435 


6 


1 & 4 


.64 


434 


9 


1 & 5 


.64 


405 


21 


1 & 6 


.70 


405 


25 


5 & 6 


.93 


405 


3 




PUPILS 


IN TREATMENTS 1, 2, 3, 4 




Cycles 


r 


N 


Number of Weeks 
Between Ratines 


1 & 2 


.73 


1683 


3 


1 & 6 


.67 


1475 


25 
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TABLE 27 

LARGE I/D MEANS DETERMINED FROM INTERACTION MATRICES 



TREATMENTS 

12 3 4 



TREATMENTS 

12 3 4 



Subjects — English Means 

for 

Fall Total 



English 

Spring 



Means 

for 

Total 



1.40 

2.22 

1.50 

2.62 


1.19 

2.29 

.83 

2.13 


.72 

2.04 

1.09 

.83 


.65 
1.08 
4.19 
. 2.64 




1.08 

2.25 

2.13 

5.27 


6.67 

1.07 

.41 

2.60 


.43 

.01 

1.48 

1.24 


.49 

.55 

5.73 

3.74 




1.94 


1.61 


1.17 


2.14 


1.72 


2.68 


2.69 


.79 


2.63 


2.20 




Math 








Math 






.66 


.36 


.72 


1.09 




.50 


.63 


.78 


1.31 




.70 


.42 


1.24 


3.77 




.43 


.15 


.99 


1.51 




.94 


.55 


.72 


.72 




1.66 


.45 


.39 


.35 




.40 


.49 


1.20 


1.61 




.66 


.91 


1.32 


1.40 




.68 


.46 


.97 


1.80 


.98 


.81 


.54 


.87 


1.14 


.84 



1.58 

.25 

.36 

.82 


S 

1.70 

.23 

.79 

.71 


Icience 

1.35 

.46 

1.10 

.69 


49 

.58 

.41 

.62 




1.20 

.35 

.42 

'.38 


Science 

.36 .95 
.16 .33 
.56 2.33 
.90 .98 


.61 

.63 

.33 

.62 




.75 


.86 


.90 


.53 


.76 


.59 


.50 


1.15 


.55 


.70 




Social Studies 




Social Studies 






.89 


.68 


1.67 


1.96 




.42 


.41 


1.61 


.95 




1.81 


.54 


.79 


.73 




.77 


.44 


1.02 


.87 




.43 


.35 


.60 


1.33 




1.09 


.28 


.92 


1.87 




1.18 


.42 


.66 


2.15 




1.14 


.21 


.51 


1.08 




1.08 


.50 


.93 


1.54 


1.01 


.86 


.34 


1.02 


1.19 


.85 


Means 


for Treatment 
















1.11 


.86 


.99 


1.50 


1.12 


1.24 


1.02 


.96 


1.38 


1.15 








(Grand Mean) 










(Grand Mean) 



appendix' h (CONTINUED) 

TABLE 28 

SMALL i/d MEANS DETERMINED FROM INTERACTION MATRICES 
TREATMENTS TREATMENTS 



1 


2 


3 


4 




1 


2 


3 


4 




Subjects — 


English 




Means 






English 




Means 










for 








for 




Fall 




Total 






Spring 




Total 


9.09 


10.81 


14.57 


12.47 




12.50 


67.00 


1.72 


9.10 




33.89 


9.25 


4.17 


13.63 




33.50 


14.33 


.50 


7.17 




3.79 


14.60 


15.25 


7.29 




60.00 


26.00 


34.00 


10.57 




2.88 


4.89 


4.15 


11.03 




7.62 


27.50 


44.00 


6.54 




12.41 


9.89 


9.54 


11.11 


10.74 


28.41 


33.71 


20.06 


8.35 


22 . 63 






Math 










Math 






31.32 


15.75 


1.54 


4.08 




11.00 


12.00 


4.44 


7.33 




3.48 


10.45 


7.45 


20.09 




4.22 


3.75 


86.00 


68.00 




4.59 


4.23 


6.29 


13.27 




9.57 


2**. 50 


28.00 


28.00 




3.38 


1.95 


13.23 


4.13 




3,88 


1.95 


13.23 


4.13 




10.69 


8.10 


7.13 


10.39 


9.08 


7.17 


10.30 


32.92 


26.87 


19.32 






Science 










Science 






3.80 


21.13 


16.50 


9.84 




3.00 


46.00 


51.00 


26.50 




9.62 


2.97 


5.36 


33.90 




44.00 


21.00 


.43 


6.86 




3.26 


14.59 


34.90 


15.59 




3.83 


10.40 


32.00 


38.00 




11.34 


7.55 


9.13 


15.38 




30.00 


4.82 


17.50 


14.33 




7.01 


11.56 


16.47 


18.68 


13.43 


20.21 


20.56 


25.23 


21.42 


21.86 






Social Studies 






Social Studies 




24.00 


21.43 


12.88 


6.38 




11.00 


27.00 


37.00 


21.00 




24,25 


9.38 


57.00 


9.42 




5.75 


36.00 


10.60 


55.00 




26.00 


8.00 


4.09 


47.50 




17.00 


8.00 


34.00 


28.00 




12.32 


8.64 


27.67 


75.50 




4.50 


28.00 


5.57 


16.00 




21.64 


11.86 


25.41 


34.70 


23.40 


9.56 


24.75 


21.79 


30.00 


21.53 


Means 


for Treatment 
















12.94 


10.35 


14.64 


18.72 


14.16 


16.34 


22.33 


25.00 


21.66 


21.33 



(Grand Mean) (Grand Mean) 
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TABLE 29 

ANALYSIS OF COVARIANCE LARGE I/D 



Source of 
Variation 


df 


X 2 


xy 


V 2 


Treatments 


3 


3.71 


2.33 


1.84 


Within 


60 


36.63 


36.24 


97.88 


Total 


63 


40.34 


38.57 


99.72 



Source of 




Sum of 


Variance 


F 


Variation 


df 


Squares 


Estimate 


Ratio 


Treatments 


3 


.81 


.27 


.257 


Within 


59 


62.03 


1.05 


___ 


Total 


62 


62,84 


— 


— 



TABLE 30 

ANALYSIS OF COVARIANCE REVISED i/d 



Source of 
Variation 


df 


X 2 


xy 


v 2 


Treatments 


3 


592.14 


25.07 


657.27 


Within 


60 


10833.03 


-4.33 


21510.08 


Total 


63 


11425.17 


20.74 


22167.35 



Source of 




Sum of 


Variance 


F 


Variation 


df 


Squares 


Estimate 


Ratio 


Treatments 


3 


657.23 


219.08 


.600 


Within 


59 


21510.08 


364.58 


...» 


Total 


62 


22167.35 


— 


— 
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TABLE 31 
MEANS FOR MTAI 
PRE-TEST 



TREATMENTS 



Subjects 

English 


1 

171 

104 

189 

108 


2 

185 

165 

114 

106 


3 

53 

159 

145 

118 


4 

56 

169 

115 

114 


5 

136 

110 

159 

149 


Means for 
Total 


Mean 


143 


142.50 


118.75 


113.50 


138.50 


131.25 


Hath 


91 


107 


94 


90 


78 






141 


160 


151 


119 


107 






131 


111 


125 


150 


126 






81 


118 


49 


141 


154 




Mean 


111 


124 


104.75 


125 


116.25 


116.20 


Science 


139 


133 


120 


95 


128 


* 




91 


179 


136 


112 


166 






124 


122 


117 


137 


157 






86 


101 


128 


73 


103 




Mean 


110 


133.75 


125.25 


104.25 


138.50 


122.35 


Social 


105 


132 


105 


160 


120 




Studies 


19 


88 


133 


102 


173 






160 


86 


160 


124 


120 






99 


107 


75 


112 


178 




Mean 


110*75 


103.25 


118.25 


124.50 


147.75 


120.90 


Meaqs for 














Treatment 


118.69 


125.88 


116.75 


116.81 


135.25 


122.68 ' 
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TABLE 32 
MEANS FOR MTAI 
POST-TEST 



TREATMENTS 















Means f d 


Subjects 


1 


2 


3 


4 


5 


Total 


English 


145 


181 


89 


49.34 


123 






127 


156 


175 


164 


118 






164 


130 


123 


91 


105 






96 


91 


127 


118 


151 





Mean 


133 


139.5 


128.50 


105.59 


124.25 


126.17 


Math 


72 


81 


95 


90 


90 






141 


154 


136 


83 


102 






116 


83 


115 


163 


140 






100 


139 


7 


126 


178 




Mean 


107.25 


114.25 


88.25 


115.50 


127.50 


110.55 



Science 


135 


101 


130 


88 


143 




106 


174 


135 


122 


147 




97 


139 


98 


105 


148 




89 


133 


95 


57 


107 


Mean 


106.75 


136.75 


114.50 


93 


136.25 117.45 



Social 


134 


126 


80 


153 


110 




Studies 


63 


113 


110 


78 


168 






145 


104 


161 


130 


116 






91 


117 


57 


125 


182 




Mean 


108.25 


115 


102 


121.50 


144 


118.15 


Mea:;s for 
Treatment 


113.81 


126.38 


108.31 


108.90 


133 


118.08 
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TABLE 33 

ANALYSIS OF VARIANCE MTAI 
PRE-TEST FOR TREATMENTS 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


78813.550 






Treatments 


4 


4059.925 


1014.981 


.952 


Subjects 


3 


2374.250 


791.417 


.742 


Interaction 


12 


8418.875 


701.573 


.658 


Within 


60. 


63960.500 


1066.008 


— 



+ 



TABLE 34 

ANALYSIS OF VARIANCE MTAI 
POST-TEST FOR TREATMENTS 


Source of 




Sum of 


Variance 


F 


Variation 


df 


Squares 


Estimate 


Ratio 


Total 


79 


88688.70 






Treatments 


4 


7829.92 


1957.48 


1.670 


Subjects 


3 


2450.05 


816.68 


.697 


Interaction 


12 


8114.06 


676.17 


.577 


Within 


60 


70294.67 


1171.58 
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TABLE 35 

ANALYSIS OF VARIANCE MTAI 
PRE-TEST FOR SUB-TREATMENTS 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


47 


49531.81 






Sub- treatments 


5 


6055.94 


1211.19 


.940 


Subjects 


3 


4358.06 


1452.69 


1.13 


Interaction 


15 


8204.31 


546.95 


.424 


Within 


24 


30913.50 


1288. C6 





TABLE 36 

ANALYSIS OF VARIANCE MTAI 
POST-TEST FOR SUB-TREATMENTS 


Source of 




Sum of 


Variance 


F 


Variation 


df 


Squares 


Estimate 


Ratio 


Total 


47 


53488.67 






Sub-treatments 


5 


5931.42 


1186.28 


.842 


Subjects 


3 


6518.17 


2172.72 


1.54 


Interaction 


15 


7215.08 


481.01 


.341 


Within 


24 


33824.00 


1409.33 
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TABLE 37 

COEFFICIENTS OF CORRELATIONS 



Achievement and Intelligence** 


Pre 


English and I.Q. 


.94 


Mathematics and I.Q. 


.95 


Social Studies and I.Q. 


.90 


Science and I.Q. 


.93 


Overall ITED and I.Q. 


.86 



STUDENT ATTITUDE (REMMERS) AND TEACHER ATTITUDE (MTAI) 



Pre Post 

-.07 .10 



ACHIEVEMENT AND ATTITUDE 



Pre 

Overall ITED and Student Attitude (Reramers) .41** 

Overall ITED and Teacher Attitude (MTAI) -.14 



OTHER POST-TEST CORRELATIONS 

Student Rating and Teacher Attitude 
Student Ratings and I/D Ratios 

Student Ratings and Principals' rating of teachers 

* Significant at the .05 level. 

** Significant at the .01 level. 

N^O for all of the above correlations 

H-26 



Po8 1 



.97 

.84 

.92 

.92 

.90 



Post 

.18 

-.35** 



r 

.28* 

-.14 

.35** 
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TEACHER REACTIONS 
FROM A STUDY BY FLOYD N. KEIM 
FORM I 

(To be completed by the teachers who received feedback on teacher- 
student verbal interaction.) 



1. What do you feel were the most rewarding features of this study 
to you? 

2. Has teacher— student verbal Interaction analysis provided you 
with new Insights Into your teaching and classroom management? 
Please give reasons for your answer. 

3. a. Do you feel that the one day of training you received In 

classrcom verbal Interaction analysis was adequate? • 

b. If you consider the training Inadequate, In what respect was 
It Inadequate? 

c. In future studies how might the training be improved? 

4. a. Do you feel that the observed teacher-student verbal inter- 

action patterns In your classroom were representative of 
your Intentions? . 

b. In what ways were they not representative of your Intentions? 



5. Do you feel that the presence of observers In your classroom 
considerably altered the normal or expected verbal Interaction 
between you and your students? 



APPENDIX I (CONTINUED) 



6. a. Do you feel that lesson content (i.e., an introductory lesson 
on some aspect of literature; review of a unit in biology) 
influenced classroom management and the. verbal interaction 
patterns exhibited during the observations by the co-invest- 
igators? 

b. If so, in what ways? 



7. a. Did you consciously attempt to change your teaching on the 
basis of the verbal interaction feedback? 

b. If so, in what ways? 



8. a. Did you experience difficulty in interpreting and understand- 

ing the feedback reports of your recorded verbal interaction 
patterns? 

b. If so, what could have been made clearer in the feedback 
reports? 

9. Would you have preferred face-to-face feedback rather than 

written feedback? Why? 

* 

10. Was the time lapse between the observations by the co~investi- 

gators ana your receipt of the feedback reports too long for you 
to derive maximum benefit ?_ . Please explain. 

11. a. Do you feel that the number of times your class was observed 

was sufficient to provide you with an index of your normal 
range of teacher-student verbal interaction patterns? 

b. If no, how frequently do you feel the observations might 
have been made to provide such an index? 

12. Did the analyses of the teacher-student verbal interaction 
patterns contained in the feedback reports suggest changes in 

your teaching behavior that you preferred not to employ? 

Please explain. 
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13. Do you have reservations about the value of verbal interaction 
analysis as it pertains directly to your teaching situation? __ 
Please explain. 



14. Do you feel that the verbal interaction patterns in the classroom are 

related to teaching effectiveness? . In what ways are they 

or are they not related? 



15. What factors do you feel limit or control the directness or indirectness 
of teaching? Please explain. 



16. What do you feel was the most annoying or objectionable part of this 
study? Please explain. 



17. From the experience gained from your participation in the feedback 
study, what suggestions would you make for the improvement of 
future studies? 



18. If you had it to do again, would you prefer not being included in a 
study such as this? Please give reasons for your answer. 
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FORM II 



(To be completed by the teachers who received feedback on the Hayes Pupil- 
Teacher Reaction Scale.) 



1. What do you feel were the most rewarding features of this study to 
you? 



2. Do you feel that the benefits of pupil-teacher ratings in improving 
teaching performance are suggicient to suggest their wide-spread use 
by high school teachers? Please explain your answer. 



3. Of what value to you was knowledge of your students' reactions to 
your teaching performance? 



A. a. On the strength of the feedback reports which contained a summary of 
the students reactions to your teaching performance, did you attempt 
to change your lesson presentations? 

b. If so, in what ways? 



5. a. Do you feel that your students' reactions to your teaching were 
reasonably accurate? __ 

b. If not, in what respect do you feel their appraisal may have been 
inaccurate? 



6. a. Do you feel that some students may have been influenced in their 
judgment by factors other than teaching performance? . 

b. If you do, please indicate factors. 

i 
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7. a. On questions contained in the Hayes Pupil-Teacher Reaction Scale in 
which youTr students rated some phase(s) of your teaching performance 
low, did you change your teaching? 

b. If so, how? 



8* Do you feel that your class is capable of evaluating your teaching 
performance? . Please give reasons for your answer. 



9. a. Did you experience difficulty in interpreting and understanding the 
feedback reports? 

b. If so, what difficulty was experienced? 



10. Would you have preferred face-to-face feedback by the co-investigators 
rather than written feedback? . Why? 



11. Do you feel that the students' rating of your teaching performance is 

a reliable index of your competency as a teacher? . Please 

give reasons for your answer. 



12. From your experience in the study, what did you like the least? 



13. What recommendations would you make to improve future studies which 
employ pupil-teacher rctings of teaching performance? 



If you had it to do again, would you prefer not being included in a 
study such as this? . Please give reasons for your answer. 



14 . 
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FORM III 



(To be completed by those teachers who received feedback on the pre-test 
subject achievement scores (I.T.E.D.) and pre-test ability scores (T.E.A.) 



1. What do you feel were the most rewarding features of this study to 



you 



2. a. Did receipt of the test scores increase your understanding of vour 
students? * . 

b. If so, in what ways? 



3. a. Was knowledge of the students’ test scores influential in changing your 
lesson plans, presentations, and classroom management? . 

b. If so, what changes did you make? 



4. a. Did your knowledge of a student's test scores influence your guidance 
and counseling of the individual student in your subject? 

b. If so. in what respect? 



5. a. Did you seek assistance from the guidance department in interpreting 
the test scores of your students? . 

b. If so, what assistance did you receive? 



6. Do you feel there is a positive relationship between the test scores 
of the students and their ratings of your teaching performance? 



7. What did you dislike most about your participation in this study? 
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From the experience gained from your participation in the feedback 
study, what suggestions would you make for the improvement of future 
studies? 



If you had it to do again, would you prefer not being included in a 
study such as this? Please give reasons for your answer. 



APPENDIX I (CONTINUED) 



Responses to the preceding questionnaires which permitted, 
a yes /no dichotomy are presented by treatment in subsequent Tables 
of this Appendix. A comparison of the teachers' responses to the 
yes /no questions contained on the questionnaires and to the same 
questions asked during the follow-up interviews, revealed that 
67 per cent of the teachers changed at least one response from 
"Yes" to "No" or "No" to "Yes". The mean number of changes in 
responses for teachers in treatment 1 through 4 was 1.49 with a 
range of changes from none to seven. The reactions of the teachers 
to questions which could not be tabulated easily or required 
further explanation than was permitted by a yes or no response, 
are presented in narrative form by treatments. 

The teachers in treatments 1, 2 and 3 who were selected 
randomly to receive. TEA and ITED test score feedback, in addition 
to feedback on teacher-student verbal interaction analysis or the 
Haves Pupil-Teacher Reaction Scale, generally found that the test 
score feedback did not add new dimensions to their understanding of 
their students since results of the schools' systematic and compre- 
hensive testing programs were already available. However, two 
major advantages of the test score feedback that the teachers 
reported were: 1. Direct receipt of test score feedback was 

convenient because it eliminated the necessity for most teachers 
to go to the guidance office and sift through cumulative records to 
obtain test score information. This encouraged greater use of the 
test scores by the teachers to determine the level of material to be 
used with the class and with individuals and to identify early in 
the year the students who might need special assistance and 
direction. 2. It permitted a comparison of TEA and ITED scores to 
be made with intelligence and achievement scores obtained through 
the use of tests published by companies other than Science Research 
Associates (Comparable results were reported by the teachers). 

TREATMENT 1 



An overwhelming majority of the teachers responded in both 
the questionnaire and the interview that the most rewarding feature 
of the study was the knowledge of the students' reaction to their 
teaching which pointed out their strengths and their weaknesses. 

Only one teacher felt there was nothing rewarding in the study. The 
following reactions are representative of the opinions of the teachers 
to the written feedback reports they received: "It gave me a self- 

analysis of my teaching and an idea of the students' opinions of 
myself as a teacher."; "Some of the criticisms, etc., that I wasn't 
aware of before, suggested changes in my teaching that would improve 
me as a teacher."; "This (feedback) has helped inspire in me a 
critical analysis of myself as a teacher,"; "This (feedback) pointed 
out weaknesses in my teaching that can be corrected. It made me 



0 
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aware of things I ^as not aware of but students were,* 1 j "I felt the 
feedback I received certainly helped me in changing my teaching 
procedures with that group.” 

one teacher in treatment 1 reported a reservation 
to the widespread use of a student-teacher rating instrument to 
improve teaching performance. Yet, twelve of the sixteen teachers 
indicated that their students might have been influenced in their 
ratings by factors other than teaching performance. Most of the 
factors mentioned were those which would tend to lower the students* 
ratings of their teachers. The most frequently recurring factors 
were i personality differences (reported ten times), marks (re- 
ported five times) , discipline problems (reported four times) and 
student attitude toward subject (reported three times). Other 
factors , that teachers felt may have influenced the student— teacher 
ratings, included previous failure with the subject, low skills, 
general scholastic ability, physical classroom conditions, student 
success in the class, and general reputation of the teachers. In 
spite of the influencing factors mentioned by the teachers, only 
three teachers of the sixteen felt that their students* ratings 
of their teaching performance were not reasonably accurate. 



An important aspect of the questionnaire was to deter- 
mine what effects the written feedback reports have upon the teach- 
ing behavior of the teachers as seen by the teachers themselves. 
Fourteen teachers reported that they changed cheir lesson present- 
ations. One of the teachers v;ho said he did not. change his 
teaching behavior was the teacher who found nothing rewarding in 
the study. The student ratings of the other teacher were suffi- 
ciently high, in his estimation, to suggest no changes in his 
teaching behavior. The latter teacher said that if students had 
not indicated that he was "one of their best teachers" dnd he had 
not "presented interesting subject matter," he "would hive changed 
drastically." The specific changes the teachers said they made 
were: improvements in the conclusions of their le^ssons, used a 

variety of teaching methods, made lesson objectives clearer, and 
provided more detailed explanations and instructions. 

Three o. : of every four teachers in treatment 1 felt 
their classes were capable of evaluating their teaching perform- 
ance. Yet only 37 per cent of the teachers felt that their 
students' ratiangs of their teaching performance were a reliable 
index of their competencies as teachers. Almost all of these teach- 
ers said that the student- teacher ratings provided only partial 
indices of their competencies and there were other factors that 
had to be considered. In addition, the teachers who held the 
opinion that teacher competence can be judged by student-teacher 
ratings had some reservations regarding the value of student- 
teacher rating devices and their implications for evaluating 
curriculum content. The reservations are summarized in the 
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following statement that was given by one of the teachers, "I feel 
that students are discerning. They are fair and objective. I feel, 
however, their judgment, as far as curriculum content is concerned, 
is not infallible by any means. They don't know, at this stage, 
what they are going to need in the way of preparation in this area.” 

To a large extent, the teachers 1 recommendations to 
improve future studies which might employ student-teacher ratings 
of teaching performance, were based upon the few aspects of the 
study that were liked least by the teachers. Two teachers felt that 
provisions should be made to permit discussion of the written feed- 
back reports with the observers. Two teachers encouraged the 
elimination of frequent administration of the Hayes Pupil-Teacher 
Reaction Scale because, ’’the students disliked answering the same 
questions." One teacher felt that the students should be better 
informed about the study and, as a result, the "student-teacner 
evaluations would be fairer and more objective — less pointed." 

It was also suggested that the teachers should be provided with a 
"printed sheet of suggestions to improve responses for each item 
on the Hayes Pupil-Teacher Reaction Scale." One teacher recommended 
that classes of students whose intelligence levels are near average 
or above average should be used in future studies for "more valid 
results." Fine illy, one of the teachers said there should be a 
follow-up study of the teacher and his classes two or three years 
from now so that comparisons of ratings could be made. 

TREATMENT 2 



The sixteen teachers who received written feedback reports 
on the verbal interactions between themselves and their students 
were asked to identify what they felt were the most rewarding 
features of the study to them. Fourteen teachers listed a variety 
of rewarding features and two teachers reported, "There were very 
few." On the positive side, two teachers replied that the feedback 
made them aware that, in some instances, they were doing too much 
lecturing and more verbal interaction could and should be used. 

Four teachers reported that knowledge of the amounts of time that 
were devoted to teacher lecture and interaction were most rewarding. 
One teacher felt that, "an awareness of what actually is taking 
place in an intergroup situation," was the highlight of the study. 

An English teacher said that, "the information, including a break- 
down of teacher-student verbal interactions, that was made available, 
. . .was most helpful in making future plans." "Self-examination of 
my methods of teaching and the attempt to improve them," was listed 
by another English teacher. A teacher with considerable classroom 
experience said, "it took me out of some complacency and it presented 
a new thought process to teaching." A social studies teacher with 
only a few years of classroom experience said, "This study made me 
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fully cognizant of the type of teacher I am. I learned a lot 
about analytical questioning and interaction analysis." The most 
rewarding feature mentioned by a teacher during the follow-up 
interview was, "the fact that it gave an opportunity for me to 
concentrate on what I was doing in terms of relationships to the 
students in the classroom. It made me think a little more 
clearly about what I was doing because, through the study, I was 
forced to do this." 

All of the teachers in this treatment indicated that 
teacher-student verbal interaction analysis provided them with 
new insights into their teaching and classroom management. Essen- 
tially, the teachers' replies centered on their recognized need to 
encourage uid obtain more interaction between themselves and their 
students and to elicit a greater amount of student involvement in 
the topics under discussion. Better than 80 per cent of the teach- 
ers either implied this need or spoke directly of the need to 
modify their teaching behavior on the basis of the written feedback 
reports they received For less than 20 per cent of the teachers, 
the written feedback reports provided a mirror image of their 
teaching and classroom management that did not necessarily imply 
change . 



Only two teachers stated that they did not consciously 
try to change their teaching behavior on the basis of the feedback 
reports they received. All of the fourteen teachers who reported 
that they did try to change their teaching behavior said they 
attempted to stimulate more student-initiated response and partici- 
pation. Eight of these teachers also said that they tried to give 
more praise and encouragement to their students. One of the teachers 
who tried to encourage more student participation in the topics under 
discussion felt that he was unsuccessful in his endeavor with the 
observed class but felt that his efforts were more fruitful in other 
classes in which he tried the same thing. He attributed his lack of 
success in the observed class to the concentration of "all the 
quiet, sedate students in this one class." 

Ten teachers felt that lesson content was a factor that 
influenced their classroom management and their recorded verbal 
interaction patterns. There was general agreement among the ten 
teachers that there was, and would be, substantially less verbal 
interaction resulting from the introduction of a new topic. A 
number of teachers not only considered lesson content as a factor 
that controls verbal interaction and classroom management, but the 
background and ability level of the student as well. 

Five teachers reported that they had difficulties in 
understanding the feedback reports of their recorded verbal inter- 
action patterns. The difficulties encountered included the inability 
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to understand the I/D and i/d ratios and the "vagueness of the 
information" contained in the feedback reports. One of the 
teachero said that the feedback reports would have been less vague 
had he really taken time to study the feedback." It was mentioned 
that face-to-face feedback would have been helpful since it would 
have provided an opportunity for discussion and interpretation of 
the information contained in the mailed feedback reports. 

Six teachers had reservations about the value of verbal 
interaction analysis as it pertained directly to their teaching 
situation. All of the six teachers felt that verbal interaction 
analysis was helpful as a yardstick to measure their teaching 
behavior but that it had its limitations. It was pointed out that 
there are so many factors which affect interaction, such as the 
subject area, the topic, the teaching behavior preferred, the 
ability of the students, etc., that the value of interaction 
analysis was restricted. A reservation strongly emphasized by two 
teachers was that verbal interaction analysis could not possibly 
measure the quality of the interaction and, as a result, its use- 
fulness to those teachers was reduced. 

r 

Teachers indicated seven factors which limited or con- 
trolled the directness or indirectness of their teaching. The 
two major factors were the ability of students and the subject 
matter being taught. The remaining factors were lesson content, 
experience and flexibility of the teacher, student attitude and 
interest, amount of material to be covered, and size of class. 

In response to the question which asked the teachers 
what was the most annoying or objectionable part of the study, 
four teachers found nothing objectionable. However, five teachers 
said they disliked the teaching schedule disruptions caused by the 
observations because they had to shift lessons to avoid conflicts 
with normally scheduled tests, visual aid presentations, laboratory 
periods, etc.. Two teachers mentioned that they were annoyed by 
the failure of their classes to respond in a normal manner while 
being observed. Another two teachers felt they lost rapport with 
their classes because of the presence of observers. One teacher 
found frustrating the inability to distinguish good teaching from 
bad teaching based on the analysis of the verbal interactions. 

Only one teacher made no recommendations which might 
improve future studies that employ verbal interaction analysis 
feedback. Five teachers emphasized the need to provide more than 
one day of orientation and training. Four teachers thought 
similar studies might be improved if the written feedback reports 
were supplemented by face-to-face feedback. A social studies 
teacher, whose observed class was below average in ability, suggest- 
ed that more than one class should be observed so that the different 
verbal teaching behaviors employed by the same teacher in classes 
of different abilities could be observed and compared. A mathematics 
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teacher suggested that tape-recording each lesson and randomly 
selecting lessons for analysis would overcome the "unnatural 
situation" that develops when observers are present in the class- 
room. Finally, a social studies teacher was hopeful that future 
studies would include some method which would permit the quality 
of verbal interactions to be recorded, measured and reported. 

TREATMENT 3 



Eleven teachers reported rewarding experiences somewhat 
similar to those mentioned by fourteen teachers in treatment 2. 

In general, the teachers were impressed by the analyses of the 
verbal interactions between themselves and their students which 
provided them with new perspectives on their patterns of teaching 
behavior. Five teachers, however, found nothing rewarding in the 
study and were critical of interaction analysis as a "singular means 
to improve teaching and learning." 

Three of the five teachers who found nothing rewarding 
in the study also felt that they gained no new insights into their 
teaching. The remaining teachers listed insights that were similar 
to those reported by the teachers in treatment 2, i.e., the need 
to encourage and obtain a greater amount of student verbal inter- 
action in the topics under discussion. 

Nine teachers, an increase of four over treatment 2, felt 
the orientation/training meeting conducted at the outset of the 
study was inadequate. The major criticisms mentioned were the time 
limitations which precluded a sufficient understanding of the 
rationale behind the study and how to interpret the materials used 
in interaction analysis. To improve the orientation and training 
of teachers in future studies which might employ interaction 
analysis, the teachers recommended that the training be spread 
over two or three days. It was further recommended by three 
teachers that time be allocated during the school year so that the 
teachers could discuss with the observers various aspects of the 
training and the study that were not readily understood. 

i 

Seven teachers who felt that the presence of observers 
considerably altered the normal or expected verbal interactions 
attributed the non-normal condition to the "goldfish bowl" uneasiness 
that affected both themselves and their students and to the "clam 
shell effect" imposed on the students by the presence of observers. 

An English teacher's comments were representative of the reactions. 
"An observer in the classro^ always results in a slightly altere * 
situation and I feel that I drive the students when there is an 
observer present. I do this because I think a segment of my mind 
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is devoted to what the observer is thinking. I know the students 
react differently. As a group they were better behaved and less 
responsive." 

To improve future studies which might employ teacher- 
student verbal interaction analysis, eight teachers recommended 
that the participants be given more training and more information 
concerning the study. Seven teachers encouraged more than three 
observations per class in order to record the normal range of 
teaching patterns. Six teachers strongly recommended that con- 
sultations or conferences be held between the observers and 
teachers so that questions concerning the interpretation of the 
feedback might readily be answered and a complete understanding 
could result. It was pointed out that face-to-face feedback could 
serve the same purposes as conferences. 

In general the teachers in treatment 3 offered more 
negative reactions to various aspects of the study and to the 
feedback they received than did the teachers assigned to other 
treatments. Yet, in spite of the negative reactions, the teachers 
praised the study for its beneficial effects upon themselves and 
their students. A biology teacher’s comment is typical of those 
received: "I've been in this teaching game long enough to know 
that you sometimes work yourself into a rut and I think that studies 
like this help you to develop a sharpened edge on teaching." 

TREATMENT 4 



The most rewarding feature of the study to ten of the 
teachers in treatment 4 had nothing to do with the feedback they 
received. Instead, they were favorably impressed by the information 
they received on teacher-student verbal interaction analysis during 
the orientation/ training meeting at the outset of the study. The 
remaining teachers felt that the receipt of the test score feedback 
was the most rewarding feature because it gave them a better under- 
standing of their pupils' abilities and interests. 

For the most part, knowledge of their students’ test 
scores was not influential in changing teaching behavior. Eight 
teachers said flatly that the test scores did not cause them to 
change their lesson plans, their presentations or their classroom 
management. Only three of the seven teachers who said they changed 
their teaching behavior to adjust to the ability of the class could 
really attribute their changes to the feedback they received. Four 
teachers who reportedly changed their teaching behavior made the 
changes as a result of the training they received in verbal inter- 
action analysis rather than the feedback. 
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The test score -feedback influenced eight teachers in 
their guidance and counseling of the indiv* iual students in their 
subjects. The guidance and counseling were limited primarily to 
those students who were unable to achieve to the level expected. 

In addition, a few teachers said they tried to be more tolerant 
and helpful to the students who were under-achieving. 

Many of the teachers felt that they received insufficient 
feedback and training to assist them in improving their teaching. 
Six teachers recommended that more observations be made and that 
feedback on the student- teacher ratings and verbal interaction 
analysis be given to the teachers immediately rather than being 
withheld until the end of the year when the feedback would be of 
less value. Six teachers suggested that more traini.ng would be 
essential in similar future studies to ‘'absorb and assimilate the 
material'* related to the study. Other teachers recommended that 
there should be more contact with the observers so that an eval- 
uation of the teaching could be made and they could "discuss 
methods which would help the teacher." 
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TABLE 38 

TABULATION OF RESPONSES OF TEACHERS IN TREATMENT I 
TO FORM II QUESTIONNAIRE (or INTERVIEW) QUESTIONS 
WHICH COULD BE ANSWERED "YES" OR "NO"* 



Question 


No . of Teachers 
Answering "Yes" 


No. of Teachers 
Answering "No" 




Other 

Responses 


2. 


15 


1 






4. a 


14 


2 




— ' 


5. a 


13 


2 


1 


("some doubt") 


6. a 


12 


4 




— 


7. a 


13 


3 




— 


8. 


12 


4 




- 


9. a 


4 


12 




- 


10. 


7 


7 


2 


("no preference") 


11. 


6 


7 


3 


("not entirely") 


14. 


2 


14 




- 



*Of the fourteen questions contained on FORM II, ten quest- 
ions could have been answered "Yes" or "No". As a reliability check 
of the responses given by the teachers to these ten questions on the 
questionnaire, the questions were asked again during the follow-up 
interview with each teacher. Among the sixteen teachers in Treat- 
ment I, there was a total of twenty changes in responses from "Yes" 
to "No" and "No" to "Yes". This averages approximately one change in 
response per teacher with a range of no changes in responses for 
nine teachers to a high of four changes for one teacher. Table 39 
contains the number of changes in responses from "Yes" to "No" and 
"No" to "Yes" to questions contained in the questionnaire which were 
asked during the interview. 



TABLE 39 

TABULATION OF CHANGES IN YES/NO RESPONSES BY TEACHERS IN TREATMENT I 
TO QUESTIONS CONTAINED ON FORM II QUESTIONNAIRE THAT WERE ASKED 

DURING THE INTERVIEWS 



Question 


Teachers Changing From 
"Yes" to "No" 


Teachers Changing 
"No" to "Yes" 


From No 
Change 


2. 


1 


2 


13 


4. a 


1 


1 


14 


5. a 


1 


1 


14 


6. a 


1 


0 


15 


7. a 


0 


0 


16 


8. 


1 


2 


13 


9. a 


0 


0 


16 


10. 


1 


3 


12 


11. 


2 


3 


11 


14. 


0 


0 


16 
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TABLE 40 

TO FORM I QUESTIONNAIRE (OR INTERVIEW) WHICH COULD BE ANSWERED "YES" 

OR "NO"* 



Question 


No. of Teachers 
Answering "Yes" 


No. of Teachers 
Answering "No" 




Other 

Responses 


2. 


16 


0 






3. a 


4 


11 


1 


("did not attend") 


4. a 


11 


4 


1 


("not completely") 


5. 


5 


11 






6. a 


10 


6 




m m 


7. a 


14 


2 




m m 


8. a 


5 


11 






9. 


9 


6 


1 


("no preference") 


10. 


I 


15 






11. a 


11 


5 






12 


0 


15 


1 


("do not know") 


13. 


6 


10 






14. 

t 


13 


0 


2 


("not entirely") 


M w 
CO 

• 


3 


13 


1 


("do not know") 



*0f the eighteen questions contained on FORM I, fourteen 
questions could have been answered "Yes" or "No". Among the sixteen 
teachers in Treatment II, there was a total of 36 changes of responses 
from Yes' to No" and "No" to "Yes". This averages approximately two 
changes in responses per teacher with a range of no changes for four 
teachers to a high of seven changes for one teacher. Table 41 contains 
the number of changes in responses from "Yes" to "No" and "No to "Yes" 
to questions contained in the questionnaire which were asked during 
the interview. 
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. TABLE 41 

TABULATION OF CHANGES IN YES/NO RESPONSES BY TEACHERS IN TREATMENT II 
TO QUESTIONS CONTAINED ON FORM I QUESTIONNAIRE 
THAT WERE ASKED DURING THE INTERVIEW 



Question 


Teachers Changing From 
"Yes" to "No" 


Teachers Changing From 
"No" to "Yes" 


No 

Change 


2. 


2 


0 


14 


3. a 


1 


0 


15 


4 • a 


2 


2 


13 


5. 


2 


1 


13 


6. a 


1 


0 


15 


7. a 


1 


1 


14 


3. a 


1 


2 


13 


9. 


0 


2 


14 


10. 


1 


1 


14 


11. a 


3 


3 


10 


12. 


0 


2 


14 


13. 


3 


0 


13 


14. 


0 


3 


12 


18. 


0 


2 


14 



TABLE 42 

TABULATION OF RESPONSES OF TEACHERS IN TREATMENT III 
TO FORM I QUESTIONNAIRE (or INTERVIEW) QUESTIONS 
WHICH COULD BE ANSWERED "YES" OR "NO"* 






No. of Teachers No. of Teachers Other 


Question 


Answering "Yes" Answering "No 


" Responses 


2. 




14 2 




3. 


a 


6 9 


1 ("did not attend") 


4. 


a 


12 4 


- 


5. 




7 9 


- 


6. 


a 


12 4 


- 


7. 


a 


12 4 


- 


8 U 


a 


9 7 


- 


9. 




13 2 


1 ("no preference") 


10. 




0 16 


*- 


11. 


a 


8 8 


- 


12. 




4 12 


- 


13. 




9 7 


- 


14. 




12 4 


- 


18. 




4 12 


- 



*Of the eighteen questions contained on FORM I t fourteen 
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questions could have been, answered "Yes" or "No". Among the sixteen 
teachers in Treatment III, there was a total of twenty-two chauges 
in responses front "Yes" to "No" and '’No" to "Yes". This averages 
approximately one change per teacher with a range of no changes for 
five teachers to a high of five changes in response for one teacher. 
Table 43 contains the number of changes in responses from "Yes" to 
"No"* and "No" to "Yes" to questions contained on the questionnaire 
which were asked during the interview. 

TABLE 43 

TABULATION OF CHANGES IN YES/NO RESPONSES BY TEACHERS IN TREATMENT II 
TO QUESTIONS CONTAINED ON FORM I QUESTIONNAIRE THAT WERE ASKED 

DURING THE INTERVIEWS 



Question 


Teachers .Changing From 
"Yes" to "No" 


Teachers Changing From 
"No" to "Yes" 


No 

Change 


2. 


1 


1 


14 


3. a 


1 


0 


15 


4. a 


0 


2 


14 


5. 


0 


0 


16 


6. a 


0 


5 


11 


7. a 


2 


0 


14 


8. a 


0 


2 


14 


9. 


0 


1 


15 


10. 


0 


0 


16 


11. a 


2 


0 


14 


12. 


2 


1 


13 


13. 


0 


1 


15 


14. 


0 


0 


16 


18. 


1 


0 


15 



TABLE 44 

TABULATION OF RESPONSES OF TEACHERS IN TREATMENT IV 
TO FORM III QUESTIONNAIRE (or INTERVIEW) QUESTIONS WHICH 
COULD BE ANSWERED "YES" OR "NO" 



Question 


No . of Teachers 
Answering "Yes" 


No. of Teachers 
Answering "No" 


Other 

Responses 


2 . a 


10 


5 




3. a 


7 


8 


- 


4, a 


8 


7 


- 


5. a 


7 


8 


- 


6. 


6 


8 


1 


9. 


2 


13 


— 



9 



O 

ERLC 
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*Of the nine questions contained on FORM III, six questions 
could have been answered “Yes" or ‘'No". Among the fifteen teachers 
in Treatment I 7, (one teacher retired late in April because of ill 
health) there was a total of thirteen changes in responses from "Yes" 
to No and No to 'Yes". This averages less than one change in 
response per teacher and ranged from no changes for eight teachers 
to a high of three changes for two teachers. Table 45 contains the 
number of changes in responses from "Yes" to "No" and "No" to "Yes" 
to questions contained in the questionnaire which were asked during 
rthe interviei*. 



TABLE 45 

TABULATION OF CHANGES IN YES /NO RESPONSES BY TEACHERS IN TREATMENT IV 
TO QUESTIONS CONTAINED ON FORM III QUESTIONNAIRE THAT WERE ASKED 

DURING THE INTERVIEWS 



Question 


Teachers Changing From 
"Yes" to "No" 


Teachers Changing From 
"No" to "Yes" 


No 

Change 


2 . a 


1 


1 


13 


3. a 


1 


1 


13 


4. a 


1 


2 


13 


5. a 


1 


0 


14 


6. 


J 


0 


12 


9. 


2 


0 


14 



TABLE 46 

TABULATION OF YES/NO RESPONSES OF TEACHERS IN TREATMENTS 1, 2 AND 3 
TO TEST SCORE FEEDBACK QUESTIONS ON FORM III QUESTIONNAIRE 





No. of Teachers 




No. of Teachers 




Answering "Yes" 




Answering "No" 




Question 


Treat. 1* 2** 3*** 


Tot. 


Treat. 1* 2** 3 *** 


Tot. 


2. a 


6 7 3 


16 


'10 4 


5 


3. a 


4 2 2 


8 


3 5 5 


13 


4. a 


6 3 4 


13 


14 3 


8 


5. a 


- 3 3 


6 


7 4 4 


15 


6. 


3 3 6 


12 


4 4 1 


9 



*One teacher reported at the conclusion of the study that the test 
score feedback that was mailed to him had not been received. 



**One teacher misplaced test results before studying them and there- 
fore could not answer the questions. 

***One teacher said that he lacked the time necessary to study the 
test scores of his students and therefore could not answer the questions. 
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